Page MenuHomeMiraheze

Decide whether to switch to opensearch as graylog seems to not be supporting versions passed 7.10
Closed, ResolvedPublic

Description

See https://www.graylog.org/post/graylog-to-add-support-for-opensearch, seems likely that graylog will drop support for es rather requiring open search.

We need to decide whether to switch to opensearch or stay with elasticsearch (although graylog doesn't support versions passed 7.10 and we're on 7.16 which has deprecated stuff so we're getting log spam. Graylog won't support version 8 either (this is because of the licensing changes).

I cannot find the deb for open search so we'll have to install from their tar.gz I think (https://opensearch.org/downloads.html)

Event Timeline

Paladox triaged this task as High priority.Apr 5 2022, 20:41
Paladox created this task.
Herald added subscribers: Unknown Object (User), Unknown Object (User), RhinosF1, Reception123. · View Herald TranscriptApr 5 2022, 20:41

Given the MW team also wish to use ES, it would make little sense to maintain two different versions of ES long term.

It would make more sense to drop graylog for another interface potentially if it will no longer support ES.

We wish to use it when the search team eventually being support up to date. By that time, it might be on open search anyway.

We wish to use it when the search team eventually being support up to date. By that time, it might be on open search anyway.

WMF are seeking to use OpenSearch instead of ES?

In T9033#183407, @John wrote:

We wish to use it when the search team eventually being support up to date. By that time, it might be on open search anyway.

WMF are seeking to use OpenSearch instead of ES?

https://phabricator.wikimedia.org/T280482

Their logstash/Kibana already does. The Search team are a major version behind anyway so they seem to be going ES6 -> ES7 -> OS.

Slowly but yes is the short answer. I don't think after this long without it an extra quarter or half will harm us.

With the above information, move to opensearch seems the better option then. @Paladox

John lowered the priority of this task from High to Normal.Apr 11 2022, 17:06
John moved this task from Incoming to Long Term on the Technology-Team (Infrastructure) board.
In T9033#183405, @John wrote:

Given the MW team also wish to use ES, it would make little sense to maintain two different versions of ES long term.

It would make more sense to drop graylog for another interface potentially if it will no longer support ES.

But which? There’s only logstash and graylog I know of, what other software could we use? (Both of those use es tho graylog only supports 7.10 and won’t support any newer). Logstash uses es and would support the newer versions.

In T9033#183409, @John wrote:

With the above information, move to opensearch seems the better option then. @Paladox

Oh I’ve just seen this sorry for the spam post above

John lowered the priority of this task from Normal to Low.Apr 17 2022, 20:34

@Paladox Is this something for us to do with switching servers?

Yeh we can do this I think (we're running a graylog version that supports OS now).

John moved this task from Backlog to Infrastructure on the Goal-2023-Jan-Jun board.

Opensearch provide a apt repo and the deb package now from 2.5 \o/

Also

root@test131:/etc/opensearch/opensearch-security# curl https://test131.miraheze.org:9200/
{
  "name" : "test131",
  "cluster_name" : "miraheze-general",
  "cluster_uuid" : "a8uCt5VDScWkZsULD9Vmnw",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.2.0",
    "build_type" : "deb",
    "build_hash" : "b1017fa3b9a1c781d4f34ecee411e0cdf930a515",
    "build_date" : "2022-08-12T15:13:15.856351Z",
    "build_snapshot" : false,
    "lucene_version" : "9.3.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
Unknown Object (User) unsubscribed.Mar 18 2023, 03:27