-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* feat: support conditionals * chore: run formatter * feat: run post-processing on the expression to avoid boxing * feat: overload traced_if for Reactant types * fix: incorrect region * fix: use `stablehlo.return_` * feat: use ExpressionsExplorer for parsing * fix: replace args * feat: compiling `if-else` blocks now work 🎉 * feat: compile scalars in branches * fix: bug in unrolling the generated code * feat: support elseif * feat: introduce a `MissingTracedValue` * feat: cleanup missing values from the IR * chore: apply formatting suggestion Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: codegen for else statements * refactor: move code to ReactantCore * chore: add to authors list * test: test if conditions * fix: check for elseif instead of not if * chore: apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * ci(buildkite): fix the workflow * test: more test cases * fix: nested if handling * fix: partial fix to mutation issue * fix: code generation * docs: rename to type instability * test: mark some tests as broken --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
- Loading branch information
1 parent
babeb7c
commit b6ee968
Showing
16 changed files
with
1,082 additions
and
29 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -81,7 +81,6 @@ dist/ | |
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
name = "Reactant" | ||
uuid = "3c362404-f566-11ee-1572-e11a4b42c853" | ||
authors = ["William Moses <[email protected]>", "Valentin Churavy <[email protected]>", "Sergio Sánchez Ramírez <[email protected]>", "Paul Berg <[email protected]>"] | ||
authors = ["William Moses <[email protected]>", "Valentin Churavy <[email protected]>", "Sergio Sánchez Ramírez <[email protected]>", "Paul Berg <[email protected]>", "Avik Pal <[email protected]>"] | ||
version = "0.2.3" | ||
|
||
[deps] | ||
|
@@ -11,6 +11,7 @@ Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9" | |
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb" | ||
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d" | ||
Preferences = "21216c6a-2e73-6563-6e65-726566657250" | ||
ReactantCore = "a3311ec8-5e00-46d5-b541-4f83e724a433" | ||
Reactant_jll = "0192cb87-2b54-54ad-80e0-3be72ad8a3c0" | ||
Scratch = "6c6a2e73-6563-6170-7368-637461726353" | ||
|
||
|
@@ -33,6 +34,7 @@ Enzyme = "0.13" | |
NNlib = "0.9" | ||
OrderedCollections = "1" | ||
Preferences = "1.4" | ||
ReactantCore = "0.1" | ||
Reactant_jll = "0.0.22" | ||
Scratch = "1.2" | ||
Statistics = "1.10" | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
MIT License | ||
|
||
Copyright (c) 2024 Enzyme Automatic Differentiation Compiler | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
name = "ReactantCore" | ||
uuid = "a3311ec8-5e00-46d5-b541-4f83e724a433" | ||
authors = ["William Moses <[email protected]>", "Valentin Churavy <[email protected]>", "Sergio Sánchez Ramírez <[email protected]>", "Paul Berg <[email protected]>", "Avik Pal <[email protected]>"] | ||
version = "0.1.0" | ||
|
||
[deps] | ||
ExpressionExplorer = "21656369-7473-754a-2065-74616d696c43" | ||
MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09" | ||
|
||
[compat] | ||
ExpressionExplorer = "1" | ||
MacroTools = "0.5.13" | ||
julia = "1.10" |
Oops, something went wrong.
b6ee968
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JuliaRegistrator register subdir=lib/ReactantCore
b6ee968
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Registration pull request created: JuliaRegistries/General/118518
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
b6ee968
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reactant.jl Benchmarks
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme)
1322156738
ns1418797944
ns0.93
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant
1293942538
ns1230657063
ns1.05
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme)
1224868312
ns1210055514
ns1.01
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme)
2323944334
ns2321453182
ns1.00
ViT base (256 x 256 x 3 x 32)/forward/CUDA/Lux
216612531
ns215031968
ns1.01
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme)
6954798003
ns5458708327
ns1.27
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant
5103509804
ns5179301625
ns0.99
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme)
5081171584
ns5152065959
ns0.99
ViT base (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme)
6720851214
ns6914653384
ns0.97
ViT base (256 x 256 x 3 x 32)/forward/CPU/Lux
36264215655
ns29634509034
ns1.22
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme)
1325295976
ns1303933391
ns1.02
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant
1316239703
ns1288941570.5
ns1.02
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme)
1223956642
ns1246884488
ns0.98
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme)
2499287891
ns2588209027
ns0.97
ViT small (256 x 256 x 3 x 4)/forward/CUDA/Lux
8665141
ns8825930
ns0.98
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme)
1575352408
ns1637260762
ns0.96
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant
1567227136
ns1607338067
ns0.98
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme)
1566092027.5
ns1592753746
ns0.98
ViT small (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme)
2878841123
ns2888392716
ns1.00
ViT small (256 x 256 x 3 x 4)/forward/CPU/Lux
2685299362
ns2959415354
ns0.91
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme)
1239206197.5
ns1320589513
ns0.94
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant
1289136308
ns1232647002.5
ns1.05
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme)
1237433180
ns1233197730.5
ns1.00
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme)
2746675598
ns2510219663
ns1.09
ViT tiny (256 x 256 x 3 x 32)/forward/CUDA/Lux
22719307
ns22686905
ns1.00
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme)
2131005675
ns2195365921
ns0.97
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant
2126128561
ns2173148463
ns0.98
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme)
2131061285
ns2160517237
ns0.99
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme)
3402150262
ns3389252115
ns1.00
ViT tiny (256 x 256 x 3 x 32)/forward/CPU/Lux
5740208504
ns5458754250.5
ns1.05
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme)
1262392264.5
ns1336344147
ns0.94
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant
1258413265.5
ns1284165465.5
ns0.98
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme)
1270552917.5
ns1264413606
ns1.00
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme)
2586048537
ns2388755659
ns1.08
ViT tiny (256 x 256 x 3 x 4)/forward/CUDA/Lux
7031315
ns7116389
ns0.99
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme)
1421898963
ns1494584041
ns0.95
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant
1430099101
ns1490742502
ns0.96
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme)
1422752680
ns1473980569
ns0.97
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme)
2655576241
ns2796807816
ns0.95
ViT tiny (256 x 256 x 3 x 4)/forward/CPU/Lux
1274970277
ns1669183460
ns0.76
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme)
1274806307.5
ns1220367359.5
ns1.04
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant
1310390497
ns1264274640.5
ns1.04
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme)
1302121842
ns1345724410.5
ns0.97
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme)
2624706431
ns2566724316
ns1.02
ViT tiny (256 x 256 x 3 x 16)/forward/CUDA/Lux
12297131
ns12278807
ns1.00
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme)
1734648342
ns1777269725
ns0.98
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant
1716005516
ns1763977334
ns0.97
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme)
1705670596
ns1773537556
ns0.96
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme)
2930317875
ns3105794746
ns0.94
ViT tiny (256 x 256 x 3 x 16)/forward/CPU/Lux
3071789485.5
ns3076042064.5
ns1.00
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme)
1351363548
ns1271346742
ns1.06
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant
1300008762
ns1246562750
ns1.04
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme)
1285804476
ns1309043330
ns0.98
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme)
2521378317
ns2442642621
ns1.03
ViT small (256 x 256 x 3 x 16)/forward/CUDA/Lux
27302342
ns27314834
ns1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme)
2243314517
ns2242544865
ns1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant
2209743795
ns2216501128
ns1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme)
2196379717
ns2196805969
ns1.00
ViT small (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme)
3417637678
ns3556647163
ns0.96
ViT small (256 x 256 x 3 x 16)/forward/CPU/Lux
5737977502
ns5559034960
ns1.03
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :after_enzyme)
1239628004
ns1242324757
ns1.00
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant
1471450378
ns1298352031
ns1.13
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :before_enzyme)
1188148551.5
ns1230035861
ns0.97
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Reactant (optimize = :only_enzyme)
2290586722
ns2637986128
ns0.87
ViT small (256 x 256 x 3 x 32)/forward/CUDA/Lux
52692914.5
ns52652664
ns1.00
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :after_enzyme)
2982530184
ns3060315768
ns0.97
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant
2990386476
ns3106069884
ns0.96
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :before_enzyme)
3011396498
ns3053865991
ns0.99
ViT small (256 x 256 x 3 x 32)/forward/CPU/Reactant (optimize = :only_enzyme)
4338309706
ns4567618226
ns0.95
ViT small (256 x 256 x 3 x 32)/forward/CPU/Lux
11645205146
ns9483960261
ns1.23
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :after_enzyme)
1216846465
ns1231844420
ns0.99
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant
1268232903.5
ns1232961844
ns1.03
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :before_enzyme)
1322945944
ns1246778659.5
ns1.06
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Reactant (optimize = :only_enzyme)
2578098657
ns2387803469
ns1.08
ViT base (256 x 256 x 3 x 16)/forward/CUDA/Lux
70862545
ns70768943
ns1.00
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :after_enzyme)
3193348347
ns3264057828
ns0.98
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant
3203590115
ns3289278023
ns0.97
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :before_enzyme)
3154476044
ns3264052473
ns0.97
ViT base (256 x 256 x 3 x 16)/forward/CPU/Reactant (optimize = :only_enzyme)
4523619517
ns4733831239
ns0.96
ViT base (256 x 256 x 3 x 16)/forward/CPU/Lux
9115055641
ns10856363466
ns0.84
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :after_enzyme)
1289173742
ns1204313355
ns1.07
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant
1268715834.5
ns1195727515.5
ns1.06
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :before_enzyme)
1269718689.5
ns1220121107.5
ns1.04
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Reactant (optimize = :only_enzyme)
2796130400
ns2407152921
ns1.16
ViT base (256 x 256 x 3 x 4)/forward/CUDA/Lux
20728567
ns20638923
ns1.00
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :after_enzyme)
1845983114
ns1946630937
ns0.95
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant
1840843333
ns1945684183
ns0.95
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :before_enzyme)
1844902802
ns1942898142
ns0.95
ViT base (256 x 256 x 3 x 4)/forward/CPU/Reactant (optimize = :only_enzyme)
3070132545
ns3272416211
ns0.94
ViT base (256 x 256 x 3 x 4)/forward/CPU/Lux
3473525524.5
ns3630039016.5
ns0.96
This comment was automatically generated by workflow using github-action-benchmark.