Upcoming end-of-life software in Spark NLP

[maziyarpanahi]

Hi,

At Spark NLP, we have always tried hard to support technologies that should have been deprecated. We have done that since these technologies move much slower in enterprise/industry, unlike personal use cases. Some of these old technologies to which we should have said goodbye are Spark/PySpark 2.3.x and 2.4.x on Java 8.

We are now communicating the end-of-life for the following technologies now that we stayed way longer than any other libraries or other managed platform so you can plan your migration to newer technologies with enough time:

What does this mean?

• From July 2022 we no longer publish Spark NLP on Maven for Spark/PySpark 2.3 including Fat JARs
• From December 2022 we no longer publish Spark NLP on Maven for Spark/PySpark 2.4 including Fat JARs
• From January 2023 we will entirely move to Java 11 and drop support for Java 8

We have made this decision by looking at all Maven downloads, usage, and consulting all platforms' life cycles such as Databricks, EMR, GCP Dataproc, etc.