Page MenuHomeMiraheze

github-migration (GitHub Migration Bot)
UserBot

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Apr 17 2016, 19:25 (430 w, 3 d)
Roles
Bot
Availability
Available
GitHub User
Unknown
Miraheze User
Unknown

Recent Activity

May 12 2020

Daily5 awarded T1: Tracking: Deployment of Miraheze a Love token.
May 12 2020, 20:24 · MediaWiki

Apr 17 2016

John closed T222: mediawiki-admins for labster as Resolved.
Apr 17 2016, 22:49 · MediaWiki
NDKilla added a comment to T216: Disaster recovery plan.

So I failfish? Check off the last two things when they're done (soon(TM)?)

https://meta.miraheze.org/wiki/Tech:Compromised_Handling

I read this and seems.. okay.. except for the fact that it more or less says "figure out how the password was compromised" or "notify operations" over and over. If mw* get's compromised I can just shut it down Kappa

Apr 17 2016, 22:49 · MediaWiki
John added a comment to T215: 504 gateway timeout trying to change email.

#430 mostly.

Apr 17 2016, 22:49 · MediaWiki
labster added a comment to T209: ClipUpload (Security).

Asking the client if I can substitute Extension:MsUpload...

Apr 17 2016, 22:49 · MediaWiki
John closed T153: Make a way so that DB's with hyphens can't be submitted as Resolved.
Apr 17 2016, 22:48 · CreateWiki
Southparkfan closed T149: Backup for izanagiwiki as Resolved.
Apr 17 2016, 22:48 · MediaWiki
John added a comment to T138: Setup backup server.

I have the bulk of the module done in my local repo. It's not committed as it breaks general puppet conventions, is very very messy and include passwords (don't worry! these are throw-away able and will be changed before deploy as there will be a lot and they'll all be in private git and template by magic and universal! where as now bacula1 has like 3 different passwords and parsoid1 has 2. I'm stopping the services on the servers).

Please don't touch bacula1.

Apr 17 2016, 22:48 · MediaWiki
revi reopened T228: SiteMatrix issue at top of the page as "Open".
Apr 17 2016, 20:49 · MediaWiki
github-migration added a comment to T229: Why is this user blocked?.

At 2016-04-14T03:49:58Z, Arcane21 wrote:

Apr 17 2016, 20:49 · Database, MediaWiki
Southparkfan added a comment to T227: Phabricator migration.

Migration is going on.

Apr 17 2016, 20:49 · MediaWiki, Phabricator, Technology-Team
labster added a comment to T229: Why is this user blocked?.

@Arcane21 ^, and also make a Phabricator account here please for the future: https://phabricator.miraheze.org/

Apr 17 2016, 20:49 · Database, MediaWiki
github-migration updated subscribers of T229: Why is this user blocked?.

@Southparkfan just an idea here but could it be the issue I the user Ant on Allthetropeswiki has the same user ID as s user who was previous indefinitely blocked?

If so, I'm calling this s non-issue in things we can control and leaving it to @labster and @Arcane21 to deal with locally however they wish to.

Thoughts?

Apr 17 2016, 20:49 · Database, MediaWiki
labster closed T228: SiteMatrix issue at top of the page as Resolved.
Apr 17 2016, 20:49 · MediaWiki
revi added a comment to T228: SiteMatrix issue at top of the page.

Whoops.

Apr 17 2016, 20:49 · MediaWiki
revi closed T228: SiteMatrix issue at top of the page as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T225: Database error on userpage.

Caused by https://github.com/miraheze/mw-config/commit/6dcad7a7267a0f8dcd1687f73f1805ef34735c67 Merged 1 1/2 months ago.

Apr 17 2016, 20:49 · MediaWiki
labster closed T224: RecentPages (Sec) as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T227: Phabricator migration.

Blockers checked off as they're not blockers anymore really.

Accounts can be created by an admin, so we don't require people to register. We don't need a misc2 for this. We don't have any wishes.

Apr 17 2016, 20:49 · MediaWiki, Phabricator, Technology-Team
Southparkfan reassigned T226: test from Southparkfan to John.
Apr 17 2016, 20:49 · MediaWiki
labster claimed T224: RecentPages (Sec).
Apr 17 2016, 20:49 · MediaWiki
labster closed T225: Database error on userpage as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T225: Database error on userpage.

Remove https://github.com/miraheze/mw-config/blob/master/LocalSettings.php#L615 to fix it please!

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T227: Phabricator migration.

I'm going to mark the 'backup local.json' thing as done as well its in git now. If that's not backed up, nothing is. Just needs an update and then to actually make it apply our version via a template.

Apr 17 2016, 20:49 · MediaWiki, Phabricator, Technology-Team
Southparkfan closed T226: test as Resolved.
Apr 17 2016, 20:49 · MediaWiki
Southparkfan claimed T226: test.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T227: Phabricator migration.

Done.

Apr 17 2016, 20:49 · MediaWiki, Phabricator, Technology-Team
labster added a comment to T224: RecentPages (Sec).

I asked the user to swap this extension out for Extension:NewestPages, which easily passed security review, and he agreed quickly. Closing this issue, as the XSS errors in RecentPages are no longer our problem.

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T227: Phabricator migration.

I intend on having security AT miraheze.org and a csr AT miraheze.org being directed to Phabricator with the intent of allowing security issues and CSR requests be literally an email away from people in case people don't want to the use the forms SPF's is making (plus an email-in security email is standard practise anyway, might as well hook it up to the task tracker).

Apr 17 2016, 20:49 · MediaWiki, Phabricator, Technology-Team
John closed T223: Install/Review FlaggedRevs as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T223: Install/Review FlaggedRevs.

Closing per #350. Without configuration data, it can't be enabled. Furthermore it doesnt need a review and installation is very strongly dependent of configuration.

I'd say employ the 'make a pull request to get this done' tactic.

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T222: mediawiki-admins for labster.

Notice: /Stage[main]/Users/Users::Hashuser[labster]/Users::User[labster]/User[labster]/ensure: created

Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T224: RecentPages (Sec).

This one fails security review. It has a pretty obvious XSS hole.

Apr 17 2016, 20:49 · MediaWiki
John closed T222: mediawiki-admins for labster as Resolved.
Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T222: mediawiki-admins for labster.
Apr 17 2016, 20:49 · MediaWiki
Southparkfan added a comment to T222: mediawiki-admins for labster.

Approved.

Apr 17 2016, 20:49 · MediaWiki
John claimed T222: mediawiki-admins for labster.
Apr 17 2016, 20:49 · MediaWiki
labster closed T221: Loops (Sec) as Resolved.
Apr 17 2016, 20:49 · MediaWiki
github-migration updated subscribers of T222: mediawiki-admins for labster.

A SSH key would be super super super useful!

@Southparkfan just give a yes here.

Apr 17 2016, 20:49 · MediaWiki
github-migration closed T220: MWException fatal exceptions as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John closed T219: Internal Error when attempting to access CA as Resolved.
Apr 17 2016, 20:49 · MediaWiki
Southparkfan assigned T221: Loops (Sec) to labster.
Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T221: Loops (Sec).

Approvified and added.

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T219: Internal Error when attempting to access CA.

Resolved. Not what I thought, some entries remained in centralauth for some reason - 2 entries exist. Which is weird as I expect 3 at minimum (Reception, the founder and myself). Seems like the script failed to remove centralauth entries for this one wiki?

Apr 17 2016, 20:49 · MediaWiki
NDKilla added a comment to T218: Deprecate SiteMatrix for CreateWiki/MM alternative.

@labster this is documented on mediawiki.org as being specific to wikimedia wikis. I saw your edit which added more technical information.

Basically we could use this but if we wanted to remove default wikis we'd need to fork the official repo.

Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
Reception123 added a comment to T219: Internal Error when attempting to access CA.

@JohnFLewis " wiki_dbname = pxgprojectwiki - can someone delete it when they read this message please?" you said something about it in chat. I think someone did something wrong there..

Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T218: Deprecate SiteMatrix for CreateWiki/MM alternative.

Upstream ticket: https://phabricator.wikimedia.org/T132125

Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
John added a comment to T219: Internal Error when attempting to access CA.

i asked as part of an on-wiki report of the issue.

It appears I typoed when dropping it from the cw_wikis table which is the issue here. No one did anything to cause it except my deletions a few days ago.

Apr 17 2016, 20:49 · MediaWiki
labster claimed T218: Deprecate SiteMatrix for CreateWiki/MM alternative.
Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
labster added a comment to T218: Deprecate SiteMatrix for CreateWiki/MM alternative.

Oh, I see why @JohnFLewis didn't fix it -- it's impossible to fix without editing the extension.json file. Thanks WMF for only writing for yourselves.

Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
John closed T218: Deprecate SiteMatrix for CreateWiki/MM alternative as Resolved.
Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
labster reopened T218: Deprecate SiteMatrix for CreateWiki/MM alternative as "Open".
Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
labster added a comment to T218: Deprecate SiteMatrix for CreateWiki/MM alternative.

It doesn't seem to have fixed it. I'm still seeing Wiktionary. https://en.wiktionary.org/wiki/nope#Adverb

https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FSiteMatrix/master/extension.json#L40 seems to list these sections. Maybe we just need to change it.

Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
Reception123 added a comment to T217: 503 Backend fetch failed..

504 Gateway Time-out and 503 Backend Fetch Failed when trying to acess the Mainpage!!
while trying to edit/save page. Tried twice.
@Southparkfan @JohnFLewis

Apr 17 2016, 20:49 · Varnish, MediaWiki
Reception123 added a comment to T217: 503 Backend fetch failed..

@Southparkfan @JohnFLewis I propose putting a global notice for this issue, as it must be annoying for users.

Apr 17 2016, 20:49 · Varnish, MediaWiki
labster added a comment to T217: 503 Backend fetch failed..

I vote for bigger problems later :P

Apr 17 2016, 20:49 · Varnish, MediaWiki
github-migration added a comment to T217: 503 Backend fetch failed..

At 2016-03-27T13:41:05Z, D66Ha wrote:

Apr 17 2016, 20:49 · Varnish, MediaWiki
Southparkfan added a comment to T217: 503 Backend fetch failed..

Account creation still is slow with php5-fpm, but unlike HHVM, account creation and changing settings actually works now.

I think that running purely php5-fpm is the temporary fix.

Apr 17 2016, 20:49 · Varnish, MediaWiki
Reception123 added a comment to T217: 503 Backend fetch failed..

Even saving pages give me 504's!! I think this issue is URGENT

Apr 17 2016, 20:49 · Varnish, MediaWiki
John assigned T217: 503 Backend fetch failed. to Southparkfan.
Apr 17 2016, 20:49 · Varnish, MediaWiki
John added a comment to T218: Deprecate SiteMatrix for CreateWiki/MM alternative.

Done.

And for reference Oromoo is a language with code om. Which is why it exists because we host a wiki with the database name omwiki.

Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
Southparkfan added a comment to T217: 503 Backend fetch failed..

So, I investigated this. This is what John and I know about this:

At random (in the sense that it's not always at 8AM or 5PM) times, NGINX begins to spill out 500 or 504 errors. mw1 throws more 504 errors than 500 errors, mw2 throws more 500 errors than 504 errors.

The 500 errors are throwed by MediaWiki, because it can't contact the MariaDB server. In some cases MariaDB just ran out of memory and needs some time to restart, in other cases the error is "Lost connection to MySQL server at 'waiting for initial communication packet', system error: 110". I'm currently looking at lowering MariaDB memory usage where possible, and I applied https://github.com/miraheze/puppet/commit/37d4a5cb26d5f660d7f01d50f227faa77dd1df59 hoping that it fixes the latter error.

The 504 error means that HHVM/PHP-FPM didn't respond in time. (Almost) all occurences of the 504 errors are because an INSERT query is a SlowTimer, or because the lock wait timeout exceeded. When one SQL query is stuck, HHVM is in a stuck state too - preventing it from serving other clients (= everyone) that do not ask for such queries being executed.

Varnish catches these errors, and depools the affected appserver(s) in an attempt to keep the site up. Unfortunately, all appservers are being depooled, with as result that Varnish throws 503 errors to everyone.

We're working on it, but the lack of collaboration with upstream (Wikimedia), RAM for db1 and extra appservers (so we can send traffic to other appservers when one or two appservers are stuck) lead into small site outages.

Apr 17 2016, 20:49 · Varnish, MediaWiki
John added a comment to T217: 503 Backend fetch failed..

As the issue is vested in MariaDB, that's not a surprise. Any active requests can generate an error (e.g. Database writes).

While saying it's urgent, the fix is a long process or alternatively hack it to work then cause bigger problems later.

Apr 17 2016, 20:49 · Varnish, MediaWiki
John claimed T218: Deprecate SiteMatrix for CreateWiki/MM alternative.
Apr 17 2016, 20:49 · WikiDiscover, Goal-2018-Jan-Jun, MacFan4000, Goal-2017-Jan-Jun, NDKilla, Configuration
Reception123 added a comment to T217: 503 Backend fetch failed..

Still happening, I think it is pretty urgent to fix it.

Apr 17 2016, 20:49 · Varnish, MediaWiki
John added a comment to T216: Disaster recovery plan.

Hardly a disaster recovery plan for seemingly when the backup issue is resolved, this is resolved now? That seems weird.

Apr 17 2016, 20:49 · MediaWiki
Southparkfan added a comment to T216: Disaster recovery plan.

Disaster recovery, crash recovery, compromised handling, .....?

Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T217: 503 Backend fetch failed..

Report via http://drunkardswalkforums.yuku.com/sreply/135505/All-The-Tropes-Wiki-Project-Part-VI

Just got some "503 Backend fetch failed" errors, on both ATT and the freebie wiki. (Luckily, this was after I finished sorting the examples and adding an image to "Signature Device" - no work was lost on my end.)

Apr 17 2016, 20:49 · Varnish, MediaWiki
John closed T216: Disaster recovery plan as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T217: 503 Backend fetch failed..

mw2 has been really really flaky the past few days it seems.

Apr 17 2016, 20:49 · Varnish, MediaWiki
github-migration updated subscribers of T216: Disaster recovery plan.

Frequent and public discussion of current server usage, security, and costs.

I'm not sure what you want by discussion. In terms of usage, we have ganglia public, and Category:Servers lists our electronic assets. This is actually a good deal more public than most orgs. Security: mostly auditable by looking at our puppet, so already mostly public. Costs: Already more public than most nonprofit orgs by having a running balance on the Finance page.

If you're asking for public discussion of which direction we should be taking on these things -- no. Or at least no more than we already do. 45% of business happens in IRC in a public logged channel, and another 50% here on Github. If you care, you're already keeping track of these places. We do not need to invite the general public to come in and bikeshed, because if they care they're already involved.

Off-wiki documentation of finances. @labster

So wait, are you asking for the some of the books to be kept secret? Or an occasional wget 'https://meta.miraheze.org/wiki/Finance'?

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T216: Disaster recovery plan.

https://meta.miraheze.org/wiki/Tech:Compromised_Handling

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T216: Disaster recovery plan.

Cron backups via maintenance scripts - bad. You'd want a fair memory use script regularly handling the work not poorly crafted maintenance scripts. Scp is also not a process at all to handle it by.

Elaborate point 3.

4 this would include full mysql backups, now that's a whole can of worms.

Puppet does all that stuff! If you have to document thing manually like how to set up a server, you've not done puppet right at all.

Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T216: Disaster recovery plan.

Is the link to the logs anywhere except the #miraheze topic? Might be nice to post it on wiki somewhere.

✅ Done

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T216: Disaster recovery plan.

Cost really is not a major public interest thing - in fact we discuss cost more than most other projects do. We shouldn't be setting a goal to plaster the cost of everything everywhere, we disclose Publically what servers are, cost is there, we discuss what upgrades are, cost is there, we even at times discuss our yearly cost, it's there. We discuss things fluidly - I dont want to force us to discuss cost rigidly.

Well mysql dumps are more or less the only private thing we'd be backing up. The whole checkbox is mysql dumps.

It is docmented! its even documented in the readme. Literally 'apt-ge install puppet' and run puppet. Bam, it's working.

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T216: Disaster recovery plan.

Hate issue lists. Anyway:

Marked off-wiki finances as done (paypal, lobster's memory - which ever way, it's not just on-wiki).

Puppet is straight forward so the 'how to set things up on a new server' is done. Obviously if db1 goes, there's no point as all the data is on it. Private git is backed up on every single server and it's not really static - it can all be changed.

Frequent discussion of thing is marked as done as, well was this ever a goal to achieve? What happens, happens. Everything can be audited (via puppet if you believe it!) and we can't force people to discuss if db1 is still needed anymore, can we? I mean - we can but it won't change anything :)

The 'discussion of private data' is an oxymoron. Like, why would you discuss what private data is, its private :) - the 'what to do is x or y is compromised, who to tell, what to say and so on' is something I'll do now.

I like to add humour.

Apr 17 2016, 20:49 · MediaWiki
NDKilla added a comment to T216: Disaster recovery plan.

45% of business happens in IRC in a public logged channel

Is the link to the logs anywhere except the #miraheze topic? Might be nice to post it on wiki somewhere.

So wait, are you asking for the some of the books to be kept secret? Or an occasional wget 'https://meta.miraheze.org/wiki/Finance'?

I don't mean keep anything secret. I mean if the site goes down, the databases get destroyed or compromised etc, it would be nice to still have a record.

Apr 17 2016, 20:49 · MediaWiki
NDKilla added a comment to T216: Disaster recovery plan.

@JohnFLewis Noted, we'd have to come up with something else.

I mean that server costs are occasionally discussed on IRC (mostly when upgrading servers) but these things should be openly discussed more often. For server usage, current tech namespace docs seem good enough.

I get full mysql backups would include private data, but I'm not exactly sure how that was relevant to that checkmark specifically.

In that case (which I assumed anyways) how to get puppet working on a brand new server should be documented

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T215: 504 gateway timeout trying to change email.

#430 mostly.

Apr 17 2016, 20:49 · MediaWiki
github-migration closed T215: 504 gateway timeout trying to change email as Resolved.
Apr 17 2016, 20:49 · MediaWiki
John closed T214: Wipe logs containing PII after 90 days as Resolved.
Apr 17 2016, 20:49 · MediaWiki
NDKilla assigned T215: 504 gateway timeout trying to change email to Southparkfan.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T214: Wipe logs containing PII after 90 days.

All NGINX logs are now removed after 84 days (https://git.io/vadiA) as opposed to nginx's default of 365 days.

Apr 17 2016, 20:49 · MediaWiki
John reassigned T214: Wipe logs containing PII after 90 days from John to Southparkfan.
Apr 17 2016, 20:49 · MediaWiki
John added a comment to T214: Wipe logs containing PII after 90 days.

Log rotation. The base should exist - just modify it.

Apr 17 2016, 20:49 · MediaWiki
John added a comment to T214: Wipe logs containing PII after 90 days.

Giving to SPF as it seems just NGINX has no log rotation at all in puppet and nginx is his area.

Apr 17 2016, 20:49 · MediaWiki
Southparkfan added a comment to T213: Conversion of dblists to JSON.

No.

Everything is in all.dblist now.

Apr 17 2016, 20:49 · CreateWiki, Configuration
John added a comment to T213: Conversion of dblists to JSON.

This breaks SiteMatrix then as it had a private lost input from private.dblist

Apr 17 2016, 20:49 · CreateWiki, Configuration
John claimed T214: Wipe logs containing PII after 90 days.
Apr 17 2016, 20:49 · MediaWiki
Southparkfan assigned T214: Wipe logs containing PII after 90 days to John.
Apr 17 2016, 20:49 · MediaWiki
labster added a comment to T212: SimpleTooltip (Security Review).

Passed and installed.

Apr 17 2016, 20:49 · MediaWiki
Southparkfan claimed T213: Conversion of dblists to JSON.
Apr 17 2016, 20:49 · CreateWiki, Configuration
github-migration updated subscribers of T213: Conversion of dblists to JSON.

This is done now.

@JohnFLewis @NDKilla I kindly ask you to monitor performance. It seems to be a bit degraded, so..

Apr 17 2016, 20:49 · CreateWiki, Configuration
John added a comment to T213: Conversion of dblists to JSON.

Does this change still write a private.dblist?

Apr 17 2016, 20:49 · CreateWiki, Configuration
labster closed T212: SimpleTooltip (Security Review) as Resolved.
Apr 17 2016, 20:49 · MediaWiki
github-migration added a comment to T211: Kill LQT for Flow.

At 2016-03-13T01:24:11Z, Arcane21 wrote:

Apr 17 2016, 20:49 · Database, Technology-Team, Configuration
NDKilla added a comment to T211: Kill LQT for Flow.

@labster no backups for you

Apr 17 2016, 20:49 · Database, Technology-Team, Configuration
labster added a comment to T211: Kill LQT for Flow.

What no I love LQT 🚰 it is best premium technology.

I'd like to have a backup of the LQT data before upgrading. Or, y'know, a full backup of ATTwiki.

Apr 17 2016, 20:49 · Database, Technology-Team, Configuration
github-migration added a comment to T211: Kill LQT for Flow.

At 2016-03-12T22:34:43Z, Arcane21 wrote:

Apr 17 2016, 20:49 · Database, Technology-Team, Configuration
Southparkfan claimed T212: SimpleTooltip (Security Review).
Apr 17 2016, 20:49 · MediaWiki