Page MenuHomeMiraheze

Create automated system for managing SSL requests
Open, NormalPublic

Description

Following the positive experience with ImportDump and given very limited experience with webservices, we've decided that for the time being it would be better to follow that model for SSL. These are the steps that are being/were taken

  • Step 1: Automatically copy and commit private keys
  • Step 2: Automatically push certificates to GitHub
  • Step 3: Create RequestSSL and automatically add ManageWiki $wgServer entry
  • Step 4: Check whether the domain is pointed via CNAME (or CNAME flattening) or if it is pointed via NS automatically add the DNS zone to the GitHub repo.
  • Step 5: Create an API endpoint on puppet181 and configure it to listen to requests from the RequestSSL extension and execute '/root/ssl-certificate' when requests are marked as completed
  • Step 6: Decommission ssl-admins

<Original plan; kept as an archive; might not follow exactly>
Stage 1:

  • Update check_reverse_dns to check records present too.
  • Move SSL generation from mwtask141 to puppet141
  • Automate copying private keys
  • Automate pushing certificates from puppet141 to GitHub

Stage 2:

  • Update certbot cli to check rDNS is correct and either CNAME or NS record is present. Add argument to skip this.

Stage 3:

  • Create a web form to automate creating SSL tasks + checking validity - refuse to create if invalid.

Stage 4:

  • create a new wrapper for generating new ssl certs, include updating ManageWiki (puppet-user will be pointless at this point).

Stage 5:

  • Move all SSL requests to the new ssl self serve site and allow one click to do everything.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
RhinosF1 triaged this task as Low priority.Jul 3 2021, 08:49
RhinosF1 updated the task description. (Show Details)

I've created a script, will add it to puppet, deploy it everywhere later.

Icinga will now go off if the domain points in a manner we don't expect.

I'll send a PR tommorow for updating ssl-certificate to run the check too.

After that, I will start work on the beta site for creating new requests. There is a varnish config PR to force all requests via jobrunner3 for it.

Unknown Object (User) moved this task from Backlog to Goals on the Technology-Team (MediaWiki) board.Jul 4 2021, 16:54
Unknown Object (User) moved this task from Backlog to MediaWiki on the Goal-2021-Jul-Dec board.Jul 31 2021, 00:28
Reception123 removed a subscriber: Unknown Object (User).Aug 24 2021, 19:05

Can we have an update on this goal please? Last update was on July 3rd.

Can we also have a plan for realistic completion or significant progress before this goal period is update December 31st please?

So far none. This is looking unlikely it'll be done this period.

Unknown Object (User) added a comment.Jan 1 2022, 17:06

I purpose that this task does not become a goal for the new goal period, as there was no progress on it last time, and does not seem that there is anyone who intends to work on it. Goals should ideally have someone fully committed to getting them done.

Unknown Object (User) moved this task from Goals to Long Term on the Technology-Team (MediaWiki) board.Jan 1 2022, 17:06
Unknown Object (User) removed a subscriber: Bongo-Cat.Jan 30 2022, 22:36
Reception123 added a subscriber: Unknown Object (User).Jan 31 2022, 18:07

I purpose that this task does not become a goal for the new goal period, as there was no progress on it last time, and does not seem that there is anyone who intends to work on it. Goals should ideally have someone fully committed to getting them done.

While this is super late, I agree that tasks should not be goals unless there is a clear and expressed desire for someone (or multiple people) to work on it. However I think it's a shame that no one wants to work on this particular task which while not urgent (since we've been doing this for 6 years now) is important to allowing for quicker resolution of SSL-related tasks.

I propose that since maybe the extent of this task is what puts people off from doing it, we start with a simpler task which would just be to automate adding private keys to puppet111 (i.e. when using the script the key is added to puppet111 automatically) therefore allowing MWEs to generate SSLs, then further automation/user-integration can be done separately.

Unknown Object (User) unsubscribed.Feb 12 2022, 07:25

Since there is no progress, I propose we simply move ssl-certificate to puppet111 (since no mw-admins have done any SSL tasks in recent years anyway) and have the private keys automatically added.

SSL certificates are now generated on puppet111 directly and private keys are automatically copied.

Unknown Object (User) subscribed.May 28 2022, 08:38
Unknown Object (User) updated the task description. (Show Details)Nov 10 2022, 16:45

Opened https://github.com/miraheze/puppet/pull/2996.

Due to the fact that the initial options seems more complicated, our currently shorter term plan would be to implement it in a similar way to ImportDump and in the end only have SRE copy/paste the command for everything.

For automatic approve = creation, that will be a longer term goal that will likely be separated out in separate tasks. For now I think it's enough of an improvement if we can get to the point where SRE members just copy a command and that's it.

While this hasn't been fully tested yet here is my not-so-perfect version of step 3 which I unfortunately didn't integrate into the ssl-certificate script due to lack of knowledge on how to do so properly and not wanting to make things too messy. Here is the current script that can be used in the meantime (not 100% sure if DNS works yet): https://phabricator.miraheze.org/P474 and hopefully be integrated into the main one soon so that step 4 can follow.

Though unofficially, adding DNS zones manually is no longer necessary!

Reception123 renamed this task from Create better system for managing SSL requests to Create automated system for managing SSL requests.Jan 20 2023, 13:20

Just so we don't forget, the current idea would be to try using https://github.com/wikimedia/acme-chief and have an API backend for ManageWiki with the web app being MediaWiki.

https://github.com/Reception123/RequestSSL has been created based on ImportDump. Things that are required to make it operational

  • Add a method so that $oldstatus and $newstatus is known in order for the updateManageWiki function to be executed only when the status is changed from something else to completed
  • Implement logging (can be copied from ManageWiki) so that when RemoteWiki is executed it logs it as if someone had changed managewiki on wiki
  • Fix issues with timestamp (might just be a problem with the SQL implemented on beta)

Due to limited knowledge on my part, it would be preferable if someone else had a go at this.

  • Check if all i18n messages make sense (can be done by anyone) [DONE!]
Unknown Object (User) unsubscribed.Mar 18 2023, 03:25
Reception123 raised the priority of this task from Low to Normal.Jan 4 2024, 17:52

Due to the large number of tasks in this area and the particular use that automation can provide, moving this task to normal priority. It'd be nice if at least the remaining fixes for RequestSSL (Step 3) can be completed soon.

Update:

Timestamps are no longer an issue. For any SRE that is a more competent developer than me (most will be!), the two remaining things for Step 3 are:

  • Add a method so that $oldstatus and $newstatus is known in order for the updateManageWiki function to be executed only when the status is changed from something else to completed
  • Implement logging so that when RemoteWiki is executed with ManageWiki it logs it as if someone had changed managewiki on wiki

(https://github.com/Reception123/RequestSSL/blob/main/includes/RequestSSLManager.php#L485)

Heads up: some custom domains are pointed using CNAME flattening. So any automation attempts should not only check for CNAME records or whether the authoritative nameservers are pointed to us, but if the A or AAAA returned points to the known IPs of cp* servers.

Heads up: some custom domains are pointed using CNAME flattening. So any automation attempts should not only check for CNAME records or whether the authoritative nameservers are pointed to us, but if the A or AAAA returned points to the known IPs of cp* servers.

Thanks for the catch. That's definitely something to keep in mind for Step 5/the python script

  • Add a method so that $oldstatus and $newstatus is known in order for the updateManageWiki function to be executed only when the status is changed from something else to completed

So, I have an idea for this, but... it involves reading the $wgRequest global. Would this be acceptable? I'll keep searching for better ways to do this anyway.

  • Add a method so that $oldstatus and $newstatus is known in order for the updateManageWiki function to be executed only when the status is changed from something else to completed

So, I have an idea for this, but... it involves reading the $wgRequest global. Would this be acceptable? I'll keep searching for better ways to do this anyway.

This is deprecated, use RequestContext::getMain()->getRequest() IIRC

UO for the rescue! I was about to commit a grave sin against best practices.

Thanks for the PR! I'll test it out shortly (I'll be busy IRL for the next two days) to confirm that it works.

Assuming it does, the only remaining blocker for Step 3 would be ManageWiki logging.

  • Implement logging so that when RemoteWiki is executed with ManageWiki it logs it as if someone had changed managewiki on wiki

Does MediaWiki even have a concept of other wikis existing other than the one currently "running"? We could open the database for the remote wiki and manually write to the logs, but I don't think that's very good.

Assuming you want to log to the remote wiki's managewiki log.

  • Implement logging so that when RemoteWiki is executed with ManageWiki it logs it as if someone had changed managewiki on wiki

Does MediaWiki even have a concept of other wikis existing other than the one currently "running"? We could open the database for the remote wiki and manually write to the logs, but I don't think that's very good.

Assuming you want to log to the remote wiki's managewiki log.

What I'd like is for it to log to ManageWiki's log on Meta, as if someone manually changed the custom domain:

06:36, 16 January 2024 Reception123 talk contribs block changed the settings "servername" for "xwiki" (Per [[Special:RequestSSLQueue/1|request]])

Thanks to @OrangeStar, RequestSSL is now functional! There's unfortunately one late issue that I thought of that will mean it can still not be made operational. For custom domains it is quite often the case that users don't point their domains properly and need guidance. My understanding is that RequestSSL uses Echo to notify users when there's a comment on their request and if they don't manually enable email notifications they might not get any and would not know that there's been a comment.

Functional but not yet over! Can't do much right now since I'm waiting for T11680 which will introduce some utility functions (To avoid reinventing the wheel) that the server program that automates cert generation would need, but I want to cleanup some of my PRs (the ones yesterday were just to make stuff work), move strings into i18n, change existing i18n strings too, remove some ID leftovers I saw while skimming the code, and automate checking that the domain is pointed correctly at least.

For this last one, I had an idea. We would create a job called RequestSSLDomainCheckJob for example, that'd be queued whenever a request is filed. This job would essentially just run a hook called onRequestSSLDomainCheck (for example) where one could write a function to check if a domain was pointed correctly, as parameter we could pass by reference the request's RequestSSLManager instance.

Reason for doing it like this is to try to keep the extension generic, and performance (to not block the form upload with DNS queries).

if they don't manually enable email notifications they might not get any and would not know that there's been a comment.

I think this shouldn't be a problem, someone involved enough with a wiki to request a custom domain paid out of their wallet will probably regularly check the request even if they don't have an email/have disabled email notifications. If you're worried about this, you could set email notifications by default for new users in mw-config (I think it is set by default actually), but there's nothing stopping anyone from filing a SSL request here and disabling email as well. Besides there's also the on-wiki Echo notification.

I also think we should strive to collect the minimum amount of information possible. I don't think we should require an email for custom domain requests. We don't for wiki requests I think.

@OrangeStar Thanks for the ideas. Indeed, it might be better to check whether the domain is pointed via PHP rather than having that in the python script and then having to contact the user afterwards and tell them it isn't. I guess what could be done then if it is not pointed is have an error display that clearly directs users to somewhere where they can get help pointing their domain.

As for responses, that is a fair point but often even with paid custom domains it seems users sometimes forget to answer, even on Phabricator where email is mandatory. Regarding "could set email notifications by default for new users in mw-config (I think it is set by default actually)" my understanding is that that is not working properly.

Thinking about it, it would still definitely be useful to check whether the domain is pointing before the request is submitting but the python script running on puppet141 would still be needed in the end in order to be able to create the DNS zone for wikis pointing NS. The script already exists but needs some adjustments.

@Reception123 I think it's about time we get a RequestSSL project and workboard on Phab. Also, add me as a member of the project if it will not have an open join policy please.

I’m not sure that ssl-admins should be decommissioned, as cert removals would still be done manually, and this would allow us to investigate should something go wrong, and also somebody might not want letsencrypt.

I’m not sure that ssl-admins should be decommissioned, as cert removals would still be done manually, and this would allow us to investigate should something go wrong, and also somebody might not want letsencrypt.

I guess it could be kept for those minor purposes as a legacy but I wouldn't see any potential SREs needing to request it as the use case would be very limited. Removals could probably be automated too and aren't that essential really.

Regarding step 4:

This is too Miraheze-specific for inclusion in the RequestSSL codebase in my opinion. It is better suited as part of T11710. In the Miraheze-specific setup of this extension, once RequestSSL sends the request to puppet, the server program receiving the request should take care of determining if we should add new DNS zones. So I think steps 4 and 5 should be merged together.

Regarding step 4:

This is too Miraheze-specific for inclusion in the RequestSSL codebase in my opinion. It is better suited as part of T11710. In the Miraheze-specific setup of this extension, once RequestSSL sends the request to puppet, the server program receiving the request should take care of determining if we should add new DNS zones. So I think steps 4 and 5 should be merged together.

Just to be clear, what you propose is the following? In my example, the domain is pointed via NS.
1: User requests SSL
2: RequestSSL checks (with puppet181's help) whether domain is pointed or not
3: RequestSSL submitted
4: ssl-certificate script once again checks whether domain is pointed and if it's pointed via NS, adds zone to GitHub
5: RequestSSL marked as completed

EDIT: in the fully automated version, steps 2 and 4 would probably be repetitive and would need merging

Regarding step 4:

This is too Miraheze-specific for inclusion in the RequestSSL codebase in my opinion. It is better suited as part of T11710. In the Miraheze-specific setup of this extension, once RequestSSL sends the request to puppet, the server program receiving the request should take care of determining if we should add new DNS zones. So I think steps 4 and 5 should be merged together.

Just to be clear, what you propose is the following? In my example, the domain is pointed via NS.
1: User requests SSL
2: RequestSSL checks (with puppet181's help) whether domain is pointed or not
3: RequestSSL submitted
4: ssl-certificate script once again checks whether domain is pointed and if it's pointed via NS, adds zone to GitHub
5: RequestSSL marked as completed

EDIT: in the fully automated version, steps 2 and 4 would probably be repetitive and would need merging

step 2 goes after step 3, actually. Also, while I don't plan on RequestSSL depending on puppet181 to do domain checks, there's nothing stopping SRE from doing that in the hook.

5 would actually go before 4, but otherwise you're correct.

@Reception123 If you want to get RequestSSL working right now, we could look at sending emails the way core does with Special:EmailUser

@Reception123 If you want to get RequestSSL working right now, we could look at sending emails the way core does with Special:EmailUser

I thought of that but since in its current form it's not *that* useful yet compared to when it will be finalized I was thinking that people wouldn't want to spend time working on a temporary solution.

I can now confirm that since notifications are fixed (thanks @Universal_Omega !) RequestSSL is operational.
What remains to be done is to add a check on-wiki for whether CNAME or NS is pointed (@Universal_Omega has an idea for how to do that easily) and then for the puppet API