Compiler returned -1 when std::vector<double> is involved

[sitbackrelax]

Start to try Enzyme in Enzyme explorer. I found it does not compile when a runtime size container is involved, e.g. std::vector, as in the following example,

double dprod(const std::vector& v) {
double r = 1;
for (int i = 0; i < v.size(); i++) {
r *= v[i];
}
return r;
}

double g(double x) {
std::vector v;
v.push_back(x);
v.push_back(x*x);
return dprod(v);
}

int main() {
double x = 3;
double d_x = __enzyme_autodiff((void*)g, enzyme_out, x);
printf("d_x = %g\n", d_x);
}

This is critical for us, because we need to use Eigen::MatrixXd and Eigen::VectorXd with their sizes determined at runtime. Their elements are functions of some physical design variables.

Thanks for your help!

[GregTheMadMonk]

Works in Enzyme Explorer: https://fwd.gymni.ch/JTva2a

[sitbackrelax]

That indeed works, thank you!

However, if I try to pass an integer as the dimension of a std::vector variable inside a function, that only works with clang18 and fails compilation with clang20. The function looks as follows,

double g2(double x, int n) {
std::vector v;
for (int i = 0; i < n; ++i) {
v.push_back(std::pow(x, i));
}
// dprod computes the production of all values inside v.
return dprod(v);
}

I tried to compute the dg2/dx as follows,
double x = 3;
int n = 5;
double dg2_dx = __enzyme_autodiff((void*)g2, enzyme_out, x, enzyme_const, n);

clang18 with "-O3 --std=c++20" works, but clang20 with "-O3 --std=c++20" fails with the following compilation error, any idea?

unknown tbaa call instruction user inst: %call.i = tail call noundef double @pow(double noundef %x, double noundef %conv.i) #12, !dbg !2031, !tbaa !2032 vdptr: {[]:Pointer, [-1]:Integer}
unknown tbaa call instruction user inst: %call.i = tail call noundef double @pow(double noundef %x, double noundef %conv.i) #13, !dbg !2030, !tbaa !2031 vdptr: {[]:Pointer, [-1]:Integer}
clang++: /root/llvm-project/llvm/lib/IR/Instruction.cpp:153: void llvm::Instruction::insertBefore(llvm::BasicBlock&, llvm::iplist_impl<llvm::simple_ilist<llvm::Instruction, llvm::ilist_iterator_bits, llvm::ilist_parentllvm::BasicBlock >, llvm::SymbolTableListTraits<llvm::Instruction, llvm::ilist_iterator_bits, llvm::ilist_parentllvm::BasicBlock > >::iterator): Assertion `!isa(this) && "Inserting PHI after debug-records!"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: /opt/compiler-explorer/clang-assertions-trunk/bin/clang++ -gdwarf-4 -g -o /tmp/compiler-explorer-compiler2024928-67-1ef00sm.0vvil/output.s -mllvm --x86-asm-syntax=intel -fno-verbose-asm -S -fpass-plugin=/opt/compiler-explorer/main/ClangEnzyme-20.so -Xclang -load -Xclang /opt/compiler-explorer/main/ClangEnzyme-20.so -fcolor-diagnostics -fno-crash-diagnostics -O3 -std=c++20

1. parser at end of file
2. Optimizer
3. Running pass "EnzymeNewPM" on module ""
   #0 0x0000000003b86708 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3b86708)
   Build a Julia compatible plugin library #1 0x0000000003b843cc llvm::sys::CleanupOnSignal(unsigned long) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3b843cc)
   Fix CMake to allow relative path and add version # to library #2 0x0000000003ad2b58 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
   Better CI for LLVM and Julia #3 0x00007fdfc09b8520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
   Fix the incredibly stupid plug AssertingVH bug #4 0x00007fdfc0a0c9fc pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0x969fc)
   Tfk maxnum #5 0x00007fdfc09b8476 gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x42476)
   Incorrect derivative on -O0 #6 0x00007fdfc099e7f3 abort (/lib/x86_64-linux-gnu/libc.so.6+0x287f3)
   Fix active variable bug #7 0x00007fdfc099e71b (/lib/x86_64-linux-gnu/libc.so.6+0x2871b)
   Compiler crash with cannot deal with ptr that isn't arg #8 0x00007fdfc09afe96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
   Handle select of pointers #9 0x000000000349f6f7 (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x349f6f7)
   Disable preprocessing optimizations for most tests #10 0x000000000349f727 llvm::Instruction::insertInto(llvm::BasicBlock*, llvm::ilist_iterator_w_bits<llvm::ilist_detail::node_options<llvm::Instruction, true, false, void, true, llvm::BasicBlock>, false, false>) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x349f727)
   Add memcpy for pointers and floats #11 0x000000000348e914 llvm::IRBuilderDefaultInserter::InsertHelper(llvm::Instruction*, llvm::Twine const&, llvm::ilist_iterator_w_bits<llvm::ilist_detail::node_options<llvm::Instruction, true, false, void, true, llvm::BasicBlock>, false, false>) const (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x348e914)
   More general for loop structure #12 0x00007fdfbfe87da3 llvm::PHINode* llvm::IRBuilderBase::Insertllvm::PHINode(llvm::PHINode*, llvm::Twine const&) const /opt/compiler-explorer/clang-assertions-trunk-20240927/include/llvm/IR/IRBuilder.h:144:22
   Cleanup files and split for organization #13 0x00007fdfbfe85bb0 llvm::IRBuilderBase::CreatePHI(llvm::Type*, unsigned int, llvm::Twine const&) /opt/compiler-explorer/clang-assertions-trunk-20240927/include/llvm/IR/IRBuilder.h:2430:3
   More loop stability (including multiple exit loops) #14 0x00007fdfc0421e2d GradientUtils::fixLCSSA(llvm::Instruction*, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:2484:76
   add functional c tests via check-enzyme-c command #15 0x00007fdfc0421fb8 GradientUtils::fixLCSSA(llvm::Instruction*, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:2493:21
   Fix calling convention and other small bugs #16 0x00007fdfc0421fb8 GradientUtils::fixLCSSA(llvm::Instruction*, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:2493:21
   Ensure increment of canonical IV dominates uses in loops #17 0x00007fdfc0421fb8 GradientUtils::fixLCSSA(llvm::Instruction*, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:2493:21
   cacheloads flag related fixes so insertsort_alt works with cacheloads=true #18 0x00007fdfc0421fb8 GradientUtils::fixLCSSA(llvm::Instruction*, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:2493:21
   Fixtfkbranch #19 0x00007fdfc0443a99 GradientUtils::lookupM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, bool, llvm::BasicBlock*) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:6605:7
   [WIP] Handle Non-outermost Dynamic Loops #20 0x00007fdfc041f4e3 GradientUtils::unwrapM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, UnwrapMode, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:2262:28
   Selective cachereads  #21 0x00007fdfc044409e GradientUtils::lookupM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, bool, llvm::BasicBlock*) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:6638:24
   Handle Triangular Loops #22 0x00007fdfc0411f34 GradientUtils::unwrapM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, UnwrapMode, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:1209:16
   add readwriteread ll test #23 0x00007fdfc044409e GradientUtils::lookupM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, bool, llvm::BasicBlock*) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:6638:24
   Global variables (no testcase yet) #24 0x00007fdfc0413266 GradientUtils::unwrapM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, UnwrapMode, llvm::BasicBlock*, bool) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:1267:19
   Differential returns that are pointer types #25 0x00007fdfc044409e GradientUtils::lookupM(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex>> const&, bool, llvm::BasicBlock*) /app/Enzyme/enzyme/Enzyme/GradientUtils.cpp:6638:24
   Active/Inactive determination (Tim, with input from Billy) #26 0x00007fdfc003885c AdjointGenerator::lookup(llvm::Value*, llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>&) /app/Enzyme/enzyme/Enzyme/AdjointGenerator.h:2138:27
   Do not do replaceFunction on a pointer return type #27 0x00007fdfc02f9d17 AdjointGenerator::visitBinaryOperator(llvm::BinaryOperator&) /tmp/build/main/clang-assertions-trunk/Enzyme/BinopDerivatives.inc:677:37
   Handle use of a better calling convention for combined forward/reverse #28 0x00007fdfc034d035 llvm::InstVisitor<AdjointGenerator, void>::visitFMul(llvm::BinaryOperator&) /opt/compiler-explorer/clang-assertions-trunk-20240927/include/llvm/IR/Instruction.def:152:1
   add differential pointer return c test #29 0x00007fdfc033da25 llvm::InstVisitor<AdjointGenerator, void>::visit(llvm::Instruction&) /opt/compiler-explorer/clang-assertions-trunk-20240927/include/llvm/IR/Instruction.def:152:1
   Better global handling and fix caching bug #30 0x00007fdfc032a0b5 llvm::InstVisitor<AdjointGenerator, void>::visit(llvm::Instruction*) /opt/compiler-explorer/clang-assertions-trunk-20240927/include/llvm/IR/InstVisitor.h:111:49
   Intsecretpointerpartialfix #31 0x00007fdfc02d27c9 EnzymeLogic::CreatePrimalAndGradient(RequestContext, ReverseCacheKey const&&, TypeAnalysis&, AugmentedReturn const*, bool) /app/Enzyme/enzyme/Enzyme/EnzymeLogic.cpp:4298:7
   Tx2 #32 0x00007fdfc024e27a (anonymous namespace)::EnzymeBase::HandleAutoDiff(llvm::Instruction*, unsigned int, llvm::Value*, llvm::Type*, llvm::SmallVectorImplllvm::Value*&, std::map<int, llvm::Type*, std::less, std::allocator<std::pair<int const, llvm::Type*>>> const&, std::vector<DIFFE_TYPE, std::allocator<DIFFE_TYPE>> const&, llvm::Function*, DerivativeMode, (anonymous namespace)::EnzymeBase::Options&, bool, llvm::SmallVectorImplllvm::CallInst*&) /app/Enzyme/enzyme/Enzyme/Enzyme.cpp:1752:46
   More advanced activity analysis and many other bugfixes #33 0x00007fdfc0250084 (anonymous namespace)::EnzymeBase::HandleAutoDiffArguments(llvm::CallInst*, DerivativeMode, bool, llvm::SmallVectorImplllvm::CallInst*&) /app/Enzyme/enzyme/Enzyme/Enzyme.cpp:2017:26
   Large working changes #34 0x00007fdfc02555a8 (anonymous namespace)::EnzymeBase::lowerEnzymeCalls(llvm::Function&, std::set<llvm::Function*, std::lessllvm::Function*, std::allocatorllvm::Function*>&) /app/Enzyme/enzyme/Enzyme/Enzyme.cpp:2775:48
   PHINode (and generally return value), doesn't presently assert failure if no arguments are active if phi is active #35 0x00007fdfc0256e9d (anonymous namespace)::EnzymeBase::run(llvm::Module&) /app/Enzyme/enzyme/Enzyme/Enzyme.cpp:3003:15
   Import Julia source and redo CI for 1.3 #36 0x00007fdfc026061c EnzymeNewPM::run(llvm::Module&, llvm::AnalysisManagerllvm::Module&) /app/Enzyme/enzyme/Enzyme/Enzyme.cpp:3264:56
   Julia test target and support for 1.3/1.4/1.5 #37 0x00007fdfc02a991f llvm::detail::PassModel<llvm::Module, EnzymeNewPM, llvm::AnalysisManagerllvm::Module>::run(llvm::Module&, llvm::AnalysisManagerllvm::Module&) /opt/compiler-explorer/clang-assertions-trunk-20240927/include/llvm/IR/PassManagerInternal.h:91:3
   How to run tests #38 0x000000000353eb80 llvm::PassManager<llvm::Module, llvm::AnalysisManagerllvm::Module>::run(llvm::Module&, llvm::AnalysisManagerllvm::Module&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x353eb80)
   What is the purpose of @preprocess_tester #39 0x0000000003e30f0b (anonymous namespace)::EmitAssemblyHelper::RunOptimizationPipeline(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_deletellvm::raw_pwrite_stream>&, std::unique_ptr<llvm::ToolOutputFile, std::default_deletellvm::ToolOutputFile>&, clang::BackendConsumer*) BackendUtil.cpp:0:0
   Julia CI agian #40 0x0000000003e344cd clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtrllvm::vfs::FileSystem, std::unique_ptr<llvm::raw_pwrite_stream, std::default_deletellvm::raw_pwrite_stream>, clang::BackendConsumer*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3e344cd)
   Benchmarks #41 0x00000000044dccde clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x44dccde)
   Start testing the Julia integration more thouroughly #42 0x00000000047d1f18 clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x47d1f18)
   Better caching behavior #43 0x00000000065fb90c clang::ParseAST(clang::Sema&, bool, bool) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x65fb90c)
   Add all of CMath to type analysis #44 0x00000000044dd0c8 clang::CodeGenAction::ExecuteAction() (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x44dd0c8)
   Fix type analysis queries for global & add bessel.jl #45 0x0000000004796d49 clang::FrontendAction::Execute() (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x4796d49)
   Optimization: more usage of combined forward/reverse pass #46 0x00000000047155ce clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x47155ce)
   Improve use of structs in calling convention & improve forward/reverse AA #47 0x000000000487c86e clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x487c86e)
   Nomulcache #48 0x0000000000cceeff cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xcceeff)
   Large Suite of Optimizations and Fixes #49 0x0000000000cc6c0a ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
   Anonymizing code #50 0x0000000004521349 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optionalllvm::StringRef>, std::__cxx11::basic_string<char, std::char_traits, std::allocator>, bool) const::'lambda'()>(long) Job.cpp:0:0
   Type Analysis of structs & deanonymize code #51 0x0000000003ad3004 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x3ad3004)
   Vc/jltests2 #52 0x000000000452193f clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optionalllvm::StringRef>, std::__cxx11::basic_string<char, std::char_traits, std::allocator>, bool) const (.part.0) Job.cpp:0:0
   Memcpy #53 0x00000000044e6ccd clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x44e6ccd)
   Prepare for upstreaming #54 0x00000000044e7dbd clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x44e7dbd)
   Remove Julia from main Enzyme repo #55 0x00000000044efa75 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0x44efa75)
   Compatibility with LLVM10 #56 0x0000000000ccbdcf clang_main(int, char**, llvm::ToolContext const&) (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xccbdcf)
   More Code Cleanup, Licensing, and Readme #57 0x0000000000b9b7f4 main (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xb9b7f4)
   Cleanup and document Type Analysis code #58 0x00007fdfc099fd90 (/lib/x86_64-linux-gnu/libc.so.6+0x29d90)
   Fix TypeAnalysis Bugs #59 0x00007fdfc099fe40 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e40)
   Activity Analysis handling of globals needs improvement #60 0x0000000000cc66be _start (/opt/compiler-explorer/clang-assertions-trunk/bin/clang+++0xcc66be)
   clang++: error: clang frontend command failed with exit code 134 (use -v to see invocation)