Skip to content

Commit

Permalink
Merge pull request modakanalytics#30 from rohithvangapalli/spark-2.4
Browse files Browse the repository at this point in the history
Updated the almaren and spark versions
  • Loading branch information
badrinathpatchikolla authored Sep 26, 2023
2 parents 9d08ca4 + 9b2c63a commit a4c66a3
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 11 deletions.
19 changes: 10 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,28 @@ For more details check the following [link](https://github.com/GoogleCloudDatapr
To add Bigquery Almaren dependency to your sbt build:

```
libraryDependencies += "com.github.music-of-the-ainur" %% "bigquery-almaren" % "0.0.7-2.4"
libraryDependencies += "com.github.music-of-the-ainur" %% "bigquery-almaren" % "0.0.8-2.4"
```

To run in spark-shell:

```
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.11:0.9.8-2.4,com.github.music-of-the-ainur:bigquery-almaren_2.11:0.0.7-2.4"
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.11:0.9.10-2.4,com.github.music-of-the-ainur:bigquery-almaren_2.11:0.0.8-2.4"
```

BigQuery Connector is available in [Maven Central](https://mvnrepository.com/artifact/com.github.music-of-the-ainur)
repository.

| version | Connector Artifact |
|----------------------------|-----------------------------------------------------------------|
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:bigquery-almaren_2.13:0.0.7-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.7-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.7-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.7-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.7-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:bigquery-almaren_2.11:0.0.7-2.4` |

| Spark 3.4.x and scala 2.13 | `com.github.music-of-the-ainur:bigquery-almaren_2.13:0.0.8-3.4` |
| Spark 3.4.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.8-3.4` |
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:bigquery-almaren_2.13:0.0.8-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.8-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.8-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.8-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:bigquery-almaren_2.12:0.0.8-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:bigquery-almaren_2.11:0.0.8-2.4` |
## Source and Target

### Source
Expand Down
4 changes: 2 additions & 2 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ scalacOptions ++= Seq("-deprecation", "-feature")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % "0.9.8-2.4" % "provided",
"com.google.cloud.spark" %% "spark-bigquery-with-dependencies" % "0.27.0",
"com.github.music-of-the-ainur" %% "almaren-framework" % "0.9.10-2.4" % "provided",
"com.google.cloud.spark" %% "spark-bigquery-with-dependencies" % "0.29.0",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.5",
"org.scalatest" %% "scalatest" % "3.2.14" % "test"
)
Expand Down

0 comments on commit a4c66a3

Please sign in to comment.