Add support for MongoDB 6.X+

[codeadict]

MongoDB 6.0 removes support for the following legacy opcodes and database commands:

• OP_INSERT

• OP_DELETE

• OP_UPDATE

• OP_KILL_CURSORS

• OP_GET_MORE

• OP_QUERY

• getLastError

MongoDB will close the connection and will not respond to the above.

See: https://www.mongodb.com/docs/v6.0/release-notes/6.0-compatibility/#legacy-opcodes-removed

[yangcancai]

1

[kjellwinblad]

Support for MongoDB 5.1+ has been added to this fork https://github.com/emqx/mongodb-erlang (MongoDB 5.1 was the first version that removed support for the legacy opcodes).

Unfortunately, it is not easy to merge this change back as EMQX's fork and this repository has diverged quite a lot. EMQX has worked on their own fork since they want to maintain hot-upgradability. I think EMQX would appreciate if someone has time/interest to merge back changes from EMQX's fork to this repository.

[codeadict]

That's awesome @kjellwinblad , i've been checking the changes and they look great. I can volunteer to merge changes back but depends on what the maintainers think about integrating all the changes the emqx fork have or only a subset of them. Thoughts @comtihon ?

[GabrielSchlatter]

1

[ioolkos]

+1 @comtihon can you comment on @codeadict's suggestion?

[comtihon]

that would be great, unfortunately I don't have time to do it myself.

[ioolkos]

@comtihon Great, thanks for your feedback! :)
@kjellwinblad, @codeadict let's see how to best organise this merge back.

[samwar]

@codeadict Any updates to provide on the progress?

[samwar]

@kjellwinblad @codeadict Any word on the changes?

If you touched a specific set of merges or modules when making the changes to support Mongo 5.1+, we could collectively backport the changes or make another fork.

[kjellwinblad]

@kjellwinblad @codeadict Any word on the changes?

If you touched a specific set of merges or modules when making the changes to support Mongo 5.1+, we could collectively backport the changes or make another fork.

Not sure what you mean with "touched a specific set of merges or modules". When I started the work to support 5.1+ in https://github.com/emqx/mongodb-erlang, the repositories had already diverged quite a bit so a merge is not straightforward. One option could be to let development continue in our (EMQX's) repository and that eventual missing features/fixes in EMQX's repository is ported from this repository. Maybe this direction is easier?

Also it might be worth mentioned that we have noticed that the current driver seems to be quite inefficient (a lot of message passing going back and forth different processes only to do a singe insert) so it might be worth it to start with a blank slate and create a new more efficient driver that only supports the new protocol.

[GabrielSchlatter]

We received notifications from MongoDB Atlas recently: "MongoDB 5.0 End of Life on October 31, 2024; automatic Atlas cluster upgrades to 6.0". Is there any chance the driver will be updated and made compatible before that date?

[comtihon]

Unfortunately I don't have time for it RN. So either someone can propose a PR or I will try to find the time for it till Oct.

[samwar]

@kjellwinblad @codeadict Any word on the changes?
If you touched a specific set of merges or modules when making the changes to support Mongo 5.1+, we could collectively backport the changes or make another fork.

Not sure what you mean with "touched a specific set of merges or modules". When I started the work to support 5.1+ in https://github.com/emqx/mongodb-erlang, the repositories had already diverged quite a bit so a merge is not straightforward. One option could be to let development continue in our (EMQX's) repository and that eventual missing features/fixes in EMQX's repository is ported from this repository. Maybe this direction is easier?

Also it might be worth mentioned that we have noticed that the current driver seems to be quite inefficient (a lot of message passing going back and forth different processes only to do a singe insert) so it might be worth it to start with a blank slate and create a new more efficient driver that only supports the new protocol.

If the drivers have diverged so much, it might not be straightforward for those of us using the OG (this) MongoDB driver to switch to using your fork. If there are still some similarities, I was hoping the changes in your driver could be picked out. If you can pinpoint a range of commits and modules where you added support for Mongo 5.1+, then it would be easier to attempt to port the changes to this driver.

It may be as easy as porting these two commits into this driver.
37a2254
c2bf978
9ea3196

[samwar]

I've started to look at the diff and port over the changes in a new fork. Hopefully I'll have this done in the next couple of weeks.

[ioolkos]

@samwar do you think porting over the changes is feasible or ETOOCOMPLEX? just interested in your assesment, no "impatience" at all.
I think projects depending on mongodb-erlang can switch to the emqx fork in any case, but I also still think bundling forces on 1 fork is the most sustainable approach for the driver.
@codeadict @kjellwinblad @GabrielSchlatter