Allow Root CA bundle configuration #1312
Comments
|
one possibility would be to add the capability to deactivate the use of |
|
If there is a libcurl setopt for this, then you should be able to do this |
|
In looking here: https://curl.se/libcurl/c/easy_setopt_options.html
|
@jswhit I think that's more or less what I'm after.
@DennisHeimbigner Are you saying these options would help me achieve this without any changes to |
|
I need to understand how what you accessing. |
|
In theory (I have never had occasion to use it), you should be able to get what you want
This should override the use of the default certificate bundle (assuming I understand the libcurl |
@DennisHeimbigner no luck with the Looking at the code in netcdf4-python/src/netCDF4/_netCDF4.pyx Lines 1313 to 1318 in c7c5f4c netcdf4-python/include/netcdf-compat.h Lines 58 to 63 in c7c5f4c /**
Set simple key=value in .rc table.
Will overwrite any existing value.
@param key
@param value
@return NC_NOERR if success
@return NC_EINVAL if fail
*/
int
nc_rc_set(const char* key, const char* value)
{
int stat = NC_NOERR;
NCglobalstate* ncg = NULL;
if(!NC_initialized) nc_initialize();
ncg = NC_getglobalstate();
assert(ncg != NULL && ncg->rcinfo != NULL && ncg->rcinfo->entries != NULL);
if(ncg->rcinfo->ignore) goto done;;
stat = NC_rcfile_insert(key,NULL,NULL,value);
done:
return stat;
}Example URL: https://thredds.rda.ucar.edu/thredds/dodsC/files/g/ds559.0/wy2016/201608/wrf2d_d01_2016-08-11_01:00:00.nc?Time[0:1:0],XLAT[150:1:550][550:1:1100],XLONG[150:1:550][550:1:1100],PREC_ACC_NC[0:1:0][150:1:550][550:1:1100] |
|
Oops. some info got lost. If you have a .pem file, my interpretation of the libcurl documentations |
|
@DennisHeimbigner no worries. I pointed |
|
One way to test is to use the curl command with your above URL and using the --cacert <file> option. |
|
I've set the environment variable Trying the same request on a different machine outside of the corporate VPN -- note the difference in |
@DennisHeimbigner again, it looks to me like the issue is |
|
I found this on stackoverflow:
|
|
I think I misunderstood your original query. You want to augment (not replace) the default certificates |
@DennisHeimbigner ultimately, yes. Haven't tested this but I think it's probably close to what I'm hoping for: if HAS_NCRCSET:
import certifi
# Use certifi CA bundle if none is configured
cert_file = os.getenv("SSL_CERT_FILE", os.getenv("CURL_CA_BUNDLE", certifi.where()))
if nc_rc_set("HTTP.SSL.CAINFO", _strencode(cert_file)) != 0:
raise RuntimeError('error setting path to SSL certificates') Currently: netcdf4-python/src/netCDF4/_netCDF4.pyx Lines 1313 to 1318 in c7c5f4c |
|
If you are augmenting, then I do not think that libcurl supports that. |
|
What I mean is that I'm effectively augmenting with a custom Root CA for my corporate VPN. In actuality I'm using a custom CA bundle created by appending my corporate VPN's cert to the certifi library's cacert.pem file. So I think what I'm talking about here still applies. @DennisHeimbigner thanks for helping sort this out. Curious what you think of the example change above. |
|
@thwllms I think your proposed change would work. Would you mind creating a PR? |
|
For some context on why netcdf4-python uses certifi and nc_rc_set to set the path to th cacert.pem file , see #1246. (basically, it was the only way to get opendap to work with wheels without patching netcdf-c). |
|
@jswhit working on a PR and got an interesting unsuccessful result. My code in # set path to SSL certificates (issue #1246)
# available starting in version 4.9.1
print(f"SSL_CERT_FILE: {os.getenv('SSL_CERT_FILE')}")
print(f"CURL_CA_BUNDLE: {os.getenv('CURL_CA_BUNDLE')}")
if HAS_NCRCSET:
import certifi
# Use certifi CA bundle if none is configured
print(f"certifi.where(): {certifi.where()}")
cert_file = os.getenv("SSL_CERT_FILE", os.getenv("CURL_CA_BUNDLE", certifi.where()))
print(f"cert_file: {cert_file}")
if nc_rc_set("HTTP.SSL.CAINFO", _strencode(cert_file)) != 0:
raise RuntimeError('error setting path to SSL certificates')diff --git a/src/netCDF4/_netCDF4.pyx b/src/netCDF4/_netCDF4.pyx
index 271a9e4a..fbfe502b 100644
--- a/src/netCDF4/_netCDF4.pyx
+++ b/src/netCDF4/_netCDF4.pyx
@@ -1312,9 +1312,15 @@ __has_ncfilter__ = HAS_NCFILTER
# set path to SSL certificates (issue #1246)
# available starting in version 4.9.1
+print(f"SSL_CERT_FILE: {os.getenv('SSL_CERT_FILE')}")
+print(f"CURL_CA_BUNDLE: {os.getenv('CURL_CA_BUNDLE')}")
if HAS_NCRCSET:
import certifi
- if nc_rc_set("HTTP.SSL.CAINFO", _strencode(certifi.where())) != 0:
+ # Use certifi CA bundle if none is configured
+ print(f"certifi.where(): {certifi.where()}")
+ cert_file = os.getenv("SSL_CERT_FILE", os.getenv("CURL_CA_BUNDLE", certifi.where()))
+ print(f"cert_file: {cert_file}")
+ if nc_rc_set("HTTP.SSL.CAINFO", _strencode(cert_file)) != 0:
raise RuntimeError('error setting path to SSL certificates')Result: It appears as though |
|
Discovered a key clue today. I'm running this in a conda environment. If I replace the following file with my correct Root CA bundle, things seem to work as expected: |
|
So it seems like the problem has to do with the conda-installed version of libcurl, which gets installed when After uninstalling the conda |
Feature request: prior to loading the
certifiCA bundle as the default, attempt to load a custom CA bundle from a sensible environment variable such asSSL_CERT_FILE.While working with
xarraybehind a corporate VPN, I had trouble connecting to the UCAR THREDDS server due to SSL errors, e.g.:I am somewhat used to working around these sorts of issues, setting environment variables like
REQUESTS_CA_BUNDLEto point to a custom CA bundle to verify certs created by the corporate VPN. Butnetcdf4did not seem to pick up those configurations. Finally I figured out thatnetcdf4was using certs directly fromcertifi(https://github.com/Unidata/netcdf4-python/blob/master/src/netCDF4/_netCDF4.pyx#L1317)Instead I switched to using the
pydapbackend, which ultimately respects the environment variableSSL_CERT_FILE:The text was updated successfully, but these errors were encountered: