Cannot expand ~

[chris-downs]

In redstorm v0.6.6 our topology dies when it hits include 'red_storm' due to the use of the tilde shortcut at this line of environment.rb:

50: DEFAULT_STORM_CONF_FILE = File.expand_path("~/.storm/storm.yaml").

Edit: To be clear, the topology submits fine from the local machine, the failure occurs inside of the JVM when storm initializes the JRuby spout and bolts.

2013-10-28 22:58:43 util [ERROR] Async loop died!
java.lang.RuntimeException: org.jruby.exceptions.RaiseException: (ArgumentError) couldn't find HOME environment -- expanding ~' at redstorm.storm.jruby.JRubyBolt.newProxyBolt(JRubyBolt.java:85) at redstorm.storm.jruby.JRubyBolt.prepare(JRubyBolt.java:42) at backtype.storm.daemon.executor$fn__4050$fn__4059.invoke(executor.clj:610) at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:679) Caused by: org.jruby.exceptions.RaiseException: (ArgumentError) couldn't find HOME environment -- expanding~'
at org.jruby.RubyFile.expand_path(org/jruby/RubyFile.java:723)

I am able to fix the issue by using an absolute path there. I understand that that isn't the correct fix since $HOME can vary between environments. I'm sure this is a configuration issue on my end or else you would have seen it as well. I just don't know how to resolve it. I've tried setting ENV['HOME'] explicitly inside of the ruby files, as well as passing in -Duser.home to the nimbus\sup\workers.

We are using Storm v0.8.2

Java environment info:
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:host.name=lt2-supervisor-i-ecca338b
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.version=1.6.0_27
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.vendor=Sun Microsystems Inc.
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.compiler=
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:os.name=Linux
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:os.arch=amd64
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:os.version=3.2.0-48-virtual
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:user.name=storm
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:user.home=/home/storm
2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:user.dir=/opt/storm/storm-0.8.2

I am unable to provide a full stack trace due to there being sensitive information in the file paths, but let me know if there is any other info I can give.

[colinsurprenant]

Thanks for the report. This is clearly an issue and this path in not even required upon topology initialization, it is only used for submission.

I guess a workaround would be to make sure there is a $HOME set in the environment running the Storm cluster. Can you verify this?

[airhorns]

@chris-downs did you end up figuring this one out?

[airhorns]

Turns out my problem was cause of runit (whoopth). Since runit descends from the init process it has no environment at all. Just had to export the HOME var before running the storm binary. We use chef to manage our runit services so here's what mine looks like:

#!/bin/sh
exec 2>&1
export HOME=/home/<%=@options[:user]%>
exec chpst -u <%=@options[:user]%> <%=@options[:install_dir]%>/bin/supervisor-control start

which when compiled looks something like

#!/bin/sh
exec 2>&1
export HOME=/home/storm
exec chpst -u storm /u/apps/storm/bin/supervisor-control start

[colinsurprenant]

eh! thanks for the followup! I will keep the issue open to remove the useless dependency on $HOME / ~ upon topology initialization.

[ismith]

I didn't notice this before I submitted #105, which should fix the bug; not sure how you want to handle combining these two issues, or just close one and merge the other.