Page MenuHomeMiraheze

Restore possibility to get image backups with DataDump
Closed, ResolvedPublic

Description

Since many users have been asking for image dumps, it would be great if the possibility to get them via DataDump could be restored soon.

Event Timeline

Reception123 triaged this task as Normal priority.Jan 12 2023, 05:39
Reception123 created this task.
Unknown Object (User) removed Unknown Object (User) as the assignee of this task.Mar 18 2023, 03:41
Unknown Object (User) removed a project: Universal Omega.
Unknown Object (User) subscribed.
Herald removed a subscriber: Unknown Object (User). · View Herald TranscriptMar 18 2023, 03:41

Since Universal Omega will no longer be working on this task, I'll provide a short explanation for why imports stop being possible. A few months ago we moved from a static-like service (GlusterFS) to Swift. Due to the way Swift operates, it is necessary to have images 'downloaded' to mwtask141 first before being zipped for DataDump. Potentially some sort of image chunking mechanism would also be necessary in order to achieve this. See https://meta.miraheze.org/wiki/Tech:Swift and other external docs for more info.

This is partially done with https://github.com/miraheze/MirahezeMagic/pull/419 but the swift upload and rm commands need to be done using DataDump backend for it to be compatible

Leaving as managewiki-restricted for now, given the potential risk of not having enough space on the server to generate the dump.