Page MenuHomeMiraheze

Automate checking that the custom domain is pointed
Open, LowPublic

Description

RequestSSL should automatically check that the custom domain is pointed correctly.

The current idea for this is to do it as follows:

  • Introduce some configs where you can specify the CNAME record, reverse DNS regex (for support of DNS providers that do domain flattening) and the authoritative namservers RequestSSL will look for.
  • Queue a job when a request is submitted.
  • Have that job compare the domain against the configured CNAME, reverse DNS records and nameservers.
    • Leave a comment with its findings.

Event Timeline

OrangeStar triaged this task as Normal priority.Jan 19 2024, 17:06
OrangeStar created this task.

@OrangeStar So then even if it's not pointing it would still create the request but just comment that it is not pointing properly? It might be nicer if it would be similar to the 'warn' feature of AbuseFilter and first stop the request and say "Your domain doesn't seem to be pointing to Miraheze, could you please check again?" and then if the user insists (i.e. clicks again) only then the request gets created.

Reception123 lowered the priority of this task from Normal to Low.Jan 19 2024, 17:10

Moving to low priority in line with other tasks as while this is a very nice feature to have it isn't 'essential'. (Note: phabricator priorities should/will be reconsidered soon)

@OrangeStar So then even if it's not pointing it would still create the request but just comment that it is not pointing properly? It might be nicer if it would be similar to the 'warn' feature of AbuseFilter and first stop the request and say "Your domain doesn't seem to be pointing to Miraheze, could you please check again?" and then if the user insists (i.e. clicks again) only then the request gets created.

This won't run during submission of the request because for us checking that it is pointed correctly means making DNS queries. For peformance we should not make synchronous/blocking network requests during form submission. So yes, this will just make a comment. Maybe even automatically reject if not pointed.

@OrangeStar So then even if it's not pointing it would still create the request but just comment that it is not pointing properly? It might be nicer if it would be similar to the 'warn' feature of AbuseFilter and first stop the request and say "Your domain doesn't seem to be pointing to Miraheze, could you please check again?" and then if the user insists (i.e. clicks again) only then the request gets created.

This won't run during submission of the request because for us checking that it is pointed correctly means making DNS queries. For peformance we should not make synchronous/blocking network requests during form submission. So yes, this will just make a comment. Maybe even automatically reject if not pointed.

That makes sense. In that case I'd prefer a new status called "Not pointing" which it would switch to. That would be a good opportunity to remove the "In progress" status as generating an SSL is a one time action so there's not really any point in marking requests as "in progress".

@OrangeStar So then even if it's not pointing it would still create the request but just comment that it is not pointing properly? It might be nicer if it would be similar to the 'warn' feature of AbuseFilter and first stop the request and say "Your domain doesn't seem to be pointing to Miraheze, could you please check again?" and then if the user insists (i.e. clicks again) only then the request gets created.

This won't run during submission of the request because for us checking that it is pointed correctly means making DNS queries. For peformance we should not make synchronous/blocking network requests during form submission. So yes, this will just make a comment. Maybe even automatically reject if not pointed.

That makes sense. In that case I'd prefer a new status called "Not pointing" which it would switch to. That would be a good opportunity to remove the "In progress" status as generating an SSL is a one time action so there's not really any point in marking requests as "in progress".

Good idea, we'll just remove that and make a new status.

OrangeStar renamed this task from Automate checking that domain is pointed automatically to Automate checking that domain is pointed.Jan 19 2024, 18:33

SQL will need regenerating with that PR.

During a conversation with @Universal_Omega on #miraheze-tech/#tech it was suggested to do these in the extension rather than using hooks. I will be rethinking my approach to this, will likely drop the proposed hook and add some more configs to RequestSSL, namely one where all subdomains under a domain can be exempted from DNS checks, and configs for CNAME and NS records that, if found, will mark the custom domain in the request as pointed.

Previous PR has been closed. I'm going to start over and leave the status changes for later (extension will just leave a comment for now). I'll be adding the configs mentioned on T11699#237771 after all.

OrangeStar renamed this task from Automate checking that domain is pointed to Automate checking that the custom domain is pointed.Mar 3 2024, 12:38
OrangeStar updated the task description. (Show Details)

CNAME and rDNS checks are being done in https://github.com/miraheze/RequestSSL/pull/40. As of writing, the job currently supports CNAME checks, rDNS checks will be done soon. Still not yet sure how to approach checking that we're the authoritative nameserver of a domain.

I know how to check if we're the authoritative nameserver for a domain now. I'm going to use RDAP. I don't like any of the libraries that exist (only one exists which I don't really like, the rest are unmaintained), so I'll make one and upload it to Packagist. @Universal_Omega would it be OK to host the repo for this library on the Miraheze GitHub org or not?

I know how to check if we're the authoritative nameserver for a domain now. I'm going to use RDAP. I don't like any of the libraries that exist (only one exists which I don't really like, the rest are unmaintained), so I'll make one and upload it to Packagist. @Universal_Omega would it be OK to host the repo for this library on the Miraheze GitHub org or not?

We probably can but is there a reason why you can't just use the built in php function, dns_get_record with DNS_NS?

For example:

$domain = 'miraheze.org';
$ns = dns_get_record( $domain, DNS_NS );

if ( $ns && isset( $ns[0]['target'] ) ) {
    $nameservers = array_column( $ns, 'target' );
    var_export( $nameservers );
}

> [
>       0 => 'ns2.miraheze.org',
>       1 => 'ns1.miraheze.org',
> ]

Unfortunately, while we can do that for CNAMEs (and will), we can't for NS.

Say you have the domain example.com, and want to point it via NS to have your Miraheze wiki there. So you point the nameservers to ns1 and ns2.

If we use dns_get_record, it will use the configured recursive nameservers for the system, which will actually ask the our nameservers when asked for the NS records for "example.com". Our nameservers don't like it when you ask them about domains we don't have a zonefile for, and will reply REFUSED.

localhost:~$ kdig @ns1.miraheze.org domainpointedwithnswithnozonefile.com
;; ->>HEADER<<- opcode: QUERY; status: REFUSED; id: 17265
;; Flags: qr rd; QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 0

;; QUESTION SECTION:
;; domainpointedwithnswithnozonefile.com. 	IN	A

;; Received 55 B
;; Time 2024-03-07 20:48:23 CET
;; From 38.46.223.204@53(UDP) in 148.9 ms

This gets masked by the recursive nameservers as a SERVFAIL, thus we can't just query the DNS for this particular check.

Unfortunately, while we can do that for CNAMEs (and will), we can't for NS.

Say you have the domain example.com, and want to point it via NS to have your Miraheze wiki there. So you point the nameservers to ns1 and ns2.

If we use dns_get_record, it will use the configured recursive nameservers for the system, which will actually ask the our nameservers when asked for the NS records for "example.com". Our nameservers don't like it when you ask them about domains we don't have a zonefile for, and will reply REFUSED.

localhost:~$ kdig @ns1.miraheze.org domainpointedwithnswithnozonefile.com
;; ->>HEADER<<- opcode: QUERY; status: REFUSED; id: 17265
;; Flags: qr rd; QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 0

;; QUESTION SECTION:
;; domainpointedwithnswithnozonefile.com. 	IN	A

;; Received 55 B
;; Time 2024-03-07 20:48:23 CET
;; From 38.46.223.204@53(UDP) in 148.9 ms

This gets masked by the recursive nameservers as a SERVFAIL, thus we can't just query the DNS for this particular check.

I came up with some albeit extremely messy code that works for me for a domain pointing at our NS but that doesn't have zone file yet:

function get_whois_server( $tld ) {
    $whois_output = '';
    $fp = fsockopen( 'whois.iana.org', 43, $errno, $errstr, 10 );
    if ( $fp ) {
        fputs( $fp, "$tld\r\n" );
        while ( !feof( $fp ) ) {
            $whois_output .= fgets( $fp, 128 );
        }
        fclose( $fp );
    }
    $matches = [];
    preg_match( '/whois:\s+(.*)/i', $whois_output, $matches );
    if ( !empty( $matches[1] ) ) {
        return trim( $matches[1] );
    } else {
        return false;
    }
}

function get_nameservers( $domain ) {
    $domain_parts = explode( '.', $domain );
    $tld = end( $domain_parts );
    $whois_server = get_whois_server( $tld );
    if ( !$whois_server ) {
        $whois_server = 'whois.verisign-grs.com';
    }
    $whois_output = '';
    $fp = fsockopen( $whois_server, 43, $errno, $errstr, 10 );
    if ( $fp ) {
        fputs( $fp, "$domain\r\n" );
        while ( !feof( $fp ) ) {
            $whois_output .= fgets( $fp, 128 );
        }
        fclose( $fp );
    }
    $nameservers = [];
    $whois_lines = explode( "\n", $whois_output );
    foreach ( $whois_lines as $line ) {
        if ( strpos( $line, 'Name Server:' ) !== false ) {
            $nameservers[] = trim( str_replace( 'Name Server:', '', $line ) );
        }
    }
    return $nameservers;
}

$domain = 'example.com';
$nameservers = get_nameservers( $domain );

if ( !empty( $nameservers ) ) {
    echo "Configured nameservers for $domain:\n";
    foreach ( $nameservers as $ns ) {
        echo "$ns\n";
    }
}

I think this could be a lot cleaner using a few MediaWiki functions though...

WHOIS unfortunately has no future; soon enough, there'll be no guarantee that gTLDs have a WHOIS server, only RDAP will be guaranteed to work with them by 2025 (https://www.icann.org/resources/pages/global-amendment-2023-en). Only problem currently is the low adoption rate of RDAP among ccTLDs since I think currently they don't have a requirement to have a RDAP server, but I think they'll follow suit sooner rather than later. RDAP also has a big advantage over WHOIS for our use case: it is machine readable JSON over HTTPS, much better than trying to parse WHOIS output (which is server-dependent). We should prepare for the future and support RDAP, therefore I still believe RequestSSL should use RDAP for this use case, and that we should write our own client library.

WHOIS unfortunately has no future; soon enough, there'll be no guarantee that gTLDs have a WHOIS server, only RDAP will be guaranteed to work with them by 2025 (https://www.icann.org/resources/pages/global-amendment-2023-en). Only problem currently is the low adoption rate of RDAP among ccTLDs since I think currently they don't have a requirement to have a RDAP server, but I think they'll follow suit sooner rather than later. RDAP also has a big advantage over WHOIS for our use case: it is machine readable JSON over HTTPS, much better than trying to parse WHOIS output (which is server-dependent). We should prepare for the future and support RDAP, therefore I still believe RequestSSL should use RDAP for this use case, and that we should write our own client library.

Then here is a similar way using RDAP, but I guess if you'd rather we could do that other method instead:

function get_registry_lookup_service( $tld ) {
    $bootstrap_url = 'https://data.iana.org/rdap/dns.json';
    $bootstrap_data = file_get_contents( $bootstrap_url );

    if ( $bootstrap_data === false ) {
        return false;
    }

    $bootstrap_json = json_decode( $bootstrap_data, true );

    foreach ( $bootstrap_json['services'] as $service ) {
        if ( in_array( $tld, $service[0] ) ) {
            return $service[1][0];
        }
    }

    return false;
}

function get_nameservers( $domain ) {
    $domain_parts = explode( '.', $domain );
    $tld = end( $domain_parts );
    $registry_url = get_registry_lookup_service( $tld );

    if ( $registry_url === false ) {
        return false;
    }

    $rdap_url = "{$registry_url}domain/$domain";

    $context = stream_context_create( [
        'http' => [
            'method' => 'GET',
            'header' => 'Accept: application/rdap+json',
        ]
    ] );

    $response = file_get_contents( $rdap_url, false, $context );

    if ( $response === false ) {
        return false;
    }

    $data = json_decode( $response, true );

    if ( isset( $data['nameservers'] ) && is_array( $data['nameservers'] ) ) {
        $nameservers = [];
        foreach ( $data['nameservers'] as $ns ) {
            $nameservers[] = $ns['ldhName'];
        }

        return $nameservers;
    } else {
        return false;
    }
}

$domain = 'example.com';
$nameservers = get_nameservers( $domain );

if ( $nameservers !== false ) {
    echo "Configured nameservers for $domain:\n";
    foreach ( $nameservers as $ns ) {
        echo "$ns\n";
    }
}