Skip to content

Commit

Permalink
feat: support conditionals (#185)
Browse files Browse the repository at this point in the history
* feat: support conditionals

* chore: run formatter

* feat: run post-processing on the expression to avoid boxing

* feat: overload traced_if for Reactant types

* fix: incorrect region

* fix: use `stablehlo.return_`

* feat: use ExpressionsExplorer for parsing

* fix: replace args

* feat: compiling `if-else` blocks now work 🎉

* feat: compile scalars in branches

* fix: bug in unrolling the generated code

* feat: support elseif

* feat: introduce a `MissingTracedValue`

* feat: cleanup missing values from the IR

* chore: apply formatting suggestion

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: codegen for else statements

* refactor: move code to ReactantCore

* chore: add to authors list

* test: test if conditions

* fix: check for elseif instead of not if

* chore: apply suggestions from code review

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* ci(buildkite): fix the workflow

* test: more test cases

* fix: nested if handling

* fix: partial fix to mutation issue

* fix: code generation

* docs: rename to type instability

* test: mark some tests as broken

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
avik-pal and github-actions[bot] authored Nov 1, 2024
1 parent babeb7c commit b6ee968
Show file tree
Hide file tree
Showing 16 changed files with 1,082 additions and 29 deletions.
15 changes: 11 additions & 4 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,20 @@ steps:
plugins:
- JuliaCI/julia#v1:
version: "{{matrix.version}}"
- JuliaCI/julia-test#v1:
test_args: "--gpu"
- JuliaCI/julia-coverage#v1:
codecov: true
dirs:
- src
- ext
- lib/ReactantCore/src
commands: |
julia --project=. -e 'println("--- :julia: Instantiating project")
using Pkg
Pkg.develop([PackageSpec(path="lib/ReactantCore")])'
julia --project=. -e 'println("--- :julia: Run Tests")
using Pkg
Pkg.test(; coverage="user")'
agents:
queue: "juliagpu"
cuda: "*"
Expand All @@ -34,7 +41,7 @@ steps:
command: |
julia --project=benchmark -e 'println("--- :julia: Instantiating project")
using Pkg
Pkg.develop([PackageSpec(path=pwd())])'
Pkg.develop([PackageSpec(path=pwd()), PackageSpec(path="lib/ReactantCore")])'
julia --project=benchmark -e 'println("--- :julia: Run Benchmarks")
include("benchmark/runbenchmarks.jl")'
Expand All @@ -59,7 +66,7 @@ steps:
command: |
julia --project=benchmark -e 'println("--- :julia: Instantiating project")
using Pkg
Pkg.develop([PackageSpec(path=pwd())])'
Pkg.develop([PackageSpec(path=pwd()), PackageSpec(path="lib/ReactantCore")])'
julia --project=benchmark -e 'println("--- :julia: Run Benchmarks")
include("benchmark/runbenchmarks.jl")'
Expand Down
21 changes: 18 additions & 3 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,19 @@ jobs:
julia --color=yes --project=deps -e 'using Pkg; Pkg.instantiate()'
SDKROOT=`xcrun --show-sdk-path` julia --color=yes --project=deps deps/build_local.jl
cp LocalPreferences.toml test/
- uses: julia-actions/julia-runtest@v1
# if: steps.buildpkg.outcome == 'success'
- name: "Install Dependencies and Run Tests"
run: |
import Pkg
Pkg.Registry.update()
# Install packages present in subdirectories
dev_pks = Pkg.PackageSpec[]
for path in ("lib/ReactantCore",)
push!(dev_pks, Pkg.PackageSpec(; path))
end
Pkg.develop(dev_pks)
Pkg.instantiate()
Pkg.test(; coverage="user")
shell: julia --color=yes --code-coverage=user --depwarn=yes --project=. {0}
id: run_tests
env:
JULIA_PKG_SERVER_REGISTRY_PREFERENCE: eager
Expand All @@ -108,7 +119,11 @@ jobs:
- run: |
julia --color=yes --project=docs -e '
using Pkg
Pkg.develop([PackageSpec(path=pwd()), PackageSpec("Reactant_jll")])
Pkg.develop([
PackageSpec(path=pwd()),
PackageSpec("Reactant_jll"),
PackageSpec(path="lib/ReactantCore")
])
Pkg.instantiate()'
env:
JULIA_PKG_SERVER_REGISTRY_PREFERENCE: eager
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
Expand Down
4 changes: 3 additions & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name = "Reactant"
uuid = "3c362404-f566-11ee-1572-e11a4b42c853"
authors = ["William Moses <[email protected]>", "Valentin Churavy <[email protected]>", "Sergio Sánchez Ramírez <[email protected]>", "Paul Berg <[email protected]>"]
authors = ["William Moses <[email protected]>", "Valentin Churavy <[email protected]>", "Sergio Sánchez Ramírez <[email protected]>", "Paul Berg <[email protected]>", "Avik Pal <[email protected]>"]
version = "0.2.3"

[deps]
Expand All @@ -11,6 +11,7 @@ Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9"
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
ReactantCore = "a3311ec8-5e00-46d5-b541-4f83e724a433"
Reactant_jll = "0192cb87-2b54-54ad-80e0-3be72ad8a3c0"
Scratch = "6c6a2e73-6563-6170-7368-637461726353"

Expand All @@ -33,6 +34,7 @@ Enzyme = "0.13"
NNlib = "0.9"
OrderedCollections = "1"
Preferences = "1.4"
ReactantCore = "0.1"
Reactant_jll = "0.0.22"
Scratch = "1.2"
Statistics = "1.10"
Expand Down
21 changes: 21 additions & 0 deletions lib/ReactantCore/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 Enzyme Automatic Differentiation Compiler

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
13 changes: 13 additions & 0 deletions lib/ReactantCore/Project.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name = "ReactantCore"
uuid = "a3311ec8-5e00-46d5-b541-4f83e724a433"
authors = ["William Moses <[email protected]>", "Valentin Churavy <[email protected]>", "Sergio Sánchez Ramírez <[email protected]>", "Paul Berg <[email protected]>", "Avik Pal <[email protected]>"]
version = "0.1.0"

[deps]
ExpressionExplorer = "21656369-7473-754a-2065-74616d696c43"
MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"

[compat]
ExpressionExplorer = "1"
MacroTools = "0.5.13"
julia = "1.10"
Loading

3 comments on commit b6ee968

@avik-pal
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register subdir=lib/ReactantCore

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/118518

Tip: Release Notes

Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.

@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReactantCore-v0.1.0 -m "<description of version>" b6ee96896bec7868b33f112c136115d12106ce80
git push origin ReactantCore-v0.1.0

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reactant.jl Benchmarks

Benchmark suite Current: b6ee968 Previous: babeb7c Ratio
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme) 1322156738 ns 1418797944 ns 0.93
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant 1293942538 ns 1230657063 ns 1.05
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme) 1224868312 ns 1210055514 ns 1.01
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme) 2323944334 ns 2321453182 ns 1.00
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Lux 216612531 ns 215031968 ns 1.01
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme) 6954798003 ns 5458708327 ns 1.27
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant 5103509804 ns 5179301625 ns 0.99
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme) 5081171584 ns 5152065959 ns 0.99
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme) 6720851214 ns 6914653384 ns 0.97
ViT base (256 x 256 x 3 x 32)/forward/CPU/Lux 36264215655 ns 29634509034 ns 1.22
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme) 1325295976 ns 1303933391 ns 1.02
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant 1316239703 ns 1288941570.5 ns 1.02
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme) 1223956642 ns 1246884488 ns 0.98
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme) 2499287891 ns 2588209027 ns 0.97
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Lux 8665141 ns 8825930 ns 0.98
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme) 1575352408 ns 1637260762 ns 0.96
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant 1567227136 ns 1607338067 ns 0.98
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme) 1566092027.5 ns 1592753746 ns 0.98
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme) 2878841123 ns 2888392716 ns 1.00
ViT small (256 x 256 x 3 x 4)/forward/CPU/Lux 2685299362 ns 2959415354 ns 0.91
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme) 1239206197.5 ns 1320589513 ns 0.94
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant 1289136308 ns 1232647002.5 ns 1.05
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme) 1237433180 ns 1233197730.5 ns 1.00
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme) 2746675598 ns 2510219663 ns 1.09
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Lux 22719307 ns 22686905 ns 1.00
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme) 2131005675 ns 2195365921 ns 0.97
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant 2126128561 ns 2173148463 ns 0.98
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme) 2131061285 ns 2160517237 ns 0.99
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme) 3402150262 ns 3389252115 ns 1.00
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Lux 5740208504 ns 5458754250.5 ns 1.05
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme) 1262392264.5 ns 1336344147 ns 0.94
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant 1258413265.5 ns 1284165465.5 ns 0.98
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme) 1270552917.5 ns 1264413606 ns 1.00
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme) 2586048537 ns 2388755659 ns 1.08
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Lux 7031315 ns 7116389 ns 0.99
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme) 1421898963 ns 1494584041 ns 0.95
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant 1430099101 ns 1490742502 ns 0.96
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme) 1422752680 ns 1473980569 ns 0.97
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme) 2655576241 ns 2796807816 ns 0.95
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Lux 1274970277 ns 1669183460 ns 0.76
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme) 1274806307.5 ns 1220367359.5 ns 1.04
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant 1310390497 ns 1264274640.5 ns 1.04
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme) 1302121842 ns 1345724410.5 ns 0.97
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme) 2624706431 ns 2566724316 ns 1.02
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Lux 12297131 ns 12278807 ns 1.00
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme) 1734648342 ns 1777269725 ns 0.98
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant 1716005516 ns 1763977334 ns 0.97
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme) 1705670596 ns 1773537556 ns 0.96
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme) 2930317875 ns 3105794746 ns 0.94
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Lux 3071789485.5 ns 3076042064.5 ns 1.00
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme) 1351363548 ns 1271346742 ns 1.06
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant 1300008762 ns 1246562750 ns 1.04
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme) 1285804476 ns 1309043330 ns 0.98
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme) 2521378317 ns 2442642621 ns 1.03
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Lux 27302342 ns 27314834 ns 1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme) 2243314517 ns 2242544865 ns 1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant 2209743795 ns 2216501128 ns 1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme) 2196379717 ns 2196805969 ns 1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme) 3417637678 ns 3556647163 ns 0.96
ViT small (256 x 256 x 3 x 16)/forward/CPU/Lux 5737977502 ns 5559034960 ns 1.03
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme) 1239628004 ns 1242324757 ns 1.00
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant 1471450378 ns 1298352031 ns 1.13
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme) 1188148551.5 ns 1230035861 ns 0.97
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme) 2290586722 ns 2637986128 ns 0.87
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Lux 52692914.5 ns 52652664 ns 1.00
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme) 2982530184 ns 3060315768 ns 0.97
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant 2990386476 ns 3106069884 ns 0.96
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme) 3011396498 ns 3053865991 ns 0.99
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme) 4338309706 ns 4567618226 ns 0.95
ViT small (256 x 256 x 3 x 32)/forward/CPU/Lux 11645205146 ns 9483960261 ns 1.23
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme) 1216846465 ns 1231844420 ns 0.99
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant 1268232903.5 ns 1232961844 ns 1.03
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme) 1322945944 ns 1246778659.5 ns 1.06
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme) 2578098657 ns 2387803469 ns 1.08
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Lux 70862545 ns 70768943 ns 1.00
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme) 3193348347 ns 3264057828 ns 0.98
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant 3203590115 ns 3289278023 ns 0.97
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme) 3154476044 ns 3264052473 ns 0.97
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme) 4523619517 ns 4733831239 ns 0.96
ViT base (256 x 256 x 3 x 16)/forward/CPU/Lux 9115055641 ns 10856363466 ns 0.84
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme) 1289173742 ns 1204313355 ns 1.07
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant 1268715834.5 ns 1195727515.5 ns 1.06
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme) 1269718689.5 ns 1220121107.5 ns 1.04
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme) 2796130400 ns 2407152921 ns 1.16
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Lux 20728567 ns 20638923 ns 1.00
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme) 1845983114 ns 1946630937 ns 0.95
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant 1840843333 ns 1945684183 ns 0.95
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme) 1844902802 ns 1942898142 ns 0.95
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme) 3070132545 ns 3272416211 ns 0.94
ViT base (256 x 256 x 3 x 4)/forward/CPU/Lux 3473525524.5 ns 3630039016.5 ns 0.96

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.