Email report generation is not working

[mdmohsin2005khan]

I have seen 2 issues with email report generation

1. email having dot(.) is considered as invalid, example: [email protected] (my office email has this pattern & when I tried generating email, I was getting invalid email in yarn logs). Tried validating this regex https://github.com/qubole/sparklens/blob/7fa57b9606abaace2ff7459028a6bf5a68fd5fa9/src/main/scala/com/qubole/sparklens/helper/EmailReportHelper.scala#L15 through https://regex101.com/ for above mentioned email and I'm getting full match as [email protected]

2. I tried generating with regular email id (without having any dots in email) and even then I was not getting any email reports.

[bharathjs93]

Same with me..

[mayurdb]

Hi, thanks for reporting the issues.

I have created the PR for the fix: #60

It would be great if you can confirm if the new regex covers the cases. Since many users must be facing an issue because of this, we will do a new release in the coming days. In the meantime, you can build the package from the source once this is merged: sbt clean assembly

[bharathjs93]

I see another issue where I'm unable to save Sparklens JSON file to S3. It gets saved only to HDFS.

[mdmohsin2005khan]

@bharathjs93

I tried giving s3a instead of just s3 and it worked for me, sparklens was writing the metrics to s3 location.

Example: --conf spark.sparklens.data.dir=s3a://bucket-name/prefixes/sparklens/