Page MenuHomeMiraheze

Added Cargo Config
Closed, ResolvedPublic

Description

May we please have the following Cargo config added to LocalSettings?
We're wanting to switch over from DPL3 to cargo, and will need to query against these.

Thank you! :)

'wgCargoPageDataColumns' => [
	'default' => [],
	'dmlwikiwiki' => [
		'creationDate',
		'modificationDate',
		'creator',
		'fullText',
		'categories',
		'numRevisions',
		'isRedirect',
	],
],

Event Timeline

Herald added subscribers: Unknown Object (User), Bongo-Cat, Unknown Object (User) and 3 others. · View Herald TranscriptNov 10 2021, 06:49
Unknown Object (User) edited projects, added Configuration; removed MediaWiki.Nov 10 2021, 07:03
Unknown Object (User) moved this task from Backlog to Short Term on the Technology-Team (MediaWiki) board.Nov 10 2021, 07:03
Unknown Object (User) moved this task from Unsorted to Short Term on the Universal Omega board.
Unknown Object (User) closed this task as Resolved.Nov 10 2021, 07:06
Unknown Object (User) claimed this task.

I see that the needed table (entitled _pageData) was indeed created by Cargo; however, it is missing columns (not able to be set via the interface only via maintenance script). I think php setCargoPageData.php needs to be run from Cargo /maintenance in order to populate the _pageData table with the columns set in that config. Found it finally in the cargo docs:

https://www.mediawiki.org/wiki/Extension:Cargo/Storing_data#Storing_page_data (below the table).

Apologies, I did not notice that was needed since I was initially following a different guide from the same extension maker that led me astray a little:
https://workingwithmediawiki.com/book/chapter16.html#toc-Section-95 (just above that bookmarked section).

Can someone please run the cargo maintenance script to populate the table as needed? Many thanks in advance!

Unknown Object (User) added a comment.Nov 24 2021, 07:28

I did run the script.

Hmm, I'm not sure what's going on then. For some reason it is not creating or populating all columns, only one column as can be seen on our private wiki here:
https://dmlwiki.miraheze.org/wiki/Special:CargoTables/_pageData

It should have the same columns in this other Miraheze wiki, of the Cargo-generated table:
https://egoish.org/wiki/Special:CargoTables/_pageData

It has identical LocalSettings.php config to us (right below us there):
https://github.com/miraheze/mw-config/blob/master/LocalSettings.php

Apologies, just to clarify, you ran both setCargoPageData.php and then cargoRecreateData.php scripts? Unfortunately, this is the one table for which there is no interface recreate data button here. Would it be possible to please run the cargoRecreateData.php again?
Thanks.

Or perhaps php cargoRecreateData.php --table _pageData might work? I'm not familiar with this table creation or functioning, didn't use it before.

Unknown Object (User) changed the task status from Invalid to Resolved.Dec 2 2021, 06:22
Unknown Object (User) added a comment.Dec 9 2021, 07:36

@CrystalClear, any reason this was reopened? If so how come?

Unknown Object (User) added a comment.Dec 9 2021, 07:37

If it was cargoRecreateData, that was ran as part of T8371.

I partially figured out what's going wrong in the initial table, with added columns not generated. After MW upgrade to 1.37, which now allows _pageData table to be regenerated via user interface, doing a replacement table (hitting refresh here is equivilent to php setCargoPageData.php --replacement). Unfortunately, this always fails part way through with the error "creating jobs to populate _pageData failed" possibly because of the number of records or run time since not done via command line?

Image 178.png (296×784 px, 14 KB)

However, the output of that failure (the incomplete replacement table) can be viewed, and it does contain the missing, desired columns, just not all rows needed. So, I believe that setCargoPageData.php again needs to be run again from command line. If that still doesn't work, there must be a bug in Cargo or something interfering with the "jobs" it mentions (though I have no clue what that exactly means). The doc didn't mention needing to do setCargoPageData.php twice, but on second run, it seems to work (if it gets all the way through).

Apologies, took me some time to get that typed out. I was in the process of installing MW 1.36.2 on my VPS, so I could try reproducing the problem there, then remembered the 1.37 upgrade, re-tested and stumbled on this.

Ah, I also just found this, potentially the extension may need to be updated if the copy tested was tested before Nov 25th?
https://www.mediawiki.org/wiki/Extension_talk:Cargo#Table_recreation_doesn't_seem_to_work_in_1.37

Yes, our version says 3.0 (ee250d4) 15:36, 22 November 2021. I'm happy to re-close this if you'd rather I create a specific task for upgrading cargo to work around the "mediawiki" issue that Yaron pointed out (he doesn't name what it is).

Unknown Object (User) added a comment.Dec 9 2021, 08:05

Ah, I also just found this, potentially the extension may need to be updated if the copy tested was tested before Nov 25th?
https://www.mediawiki.org/wiki/Extension_talk:Cargo#Table_recreation_doesn't_seem_to_work_in_1.37

It does look like Cargo needs to be updated to fix that.

Thanks for checking, strange he noted it was a MW 1.37 bug (though just glad he added a workaround in Cargo). I'll have to wander blindly around mediawiki phab to see if I can find where the bug is mentioned.

Found:
https://phabricator.wikimedia.org/rECRG8f0c05eb211cb6dd0ecb3eb56f69138d09ca03df

And:
https://phabricator.wikimedia.org/T296435

Unknown Object (User) closed this task as Resolved.Dec 10 2021, 22:27