Inconsistent variable labels when reading sas7bdat and xpt

[guoranba]

Hi there,

I got inconsistent variable labels when reading SAS sas7bdat and xpt file. Please see the sample code and corresponding result below for details.

Sample code:

import pyreadstat as prs  
ori,oriinfo=prs.read_sas7bdat('ads.sas7bdat',encoding="gbk",metadataonly=True)  
prs.write_xport(df=ori,dst_path="ads.xpt",file_label=oriinfo.file_label,column_labels=oriinfo.column_names_to_labels,
                table_name='ADS',file_format_version=5)
val,valinfo=prs.read_xport("ads.xpt",encoding="utf-8",metadataonly=True)
oriinfo.column_names_to_labels.get("MHTERM")
valinfo.column_names_to_labels.get("MHTERM")

Result:

It seems the label of variable MHTERM generated by read_xport has been truncated. Anybody could hep me with this? Thanks!
ads.zip

[ofajardo]

Probably it is because the maximum length of a variable label in an XPT file is 40 characters:
see here.

full = oriinfo.column_names_to_labels.get("MHTERM")
cut = valinfo.column_names_to_labels.get("MHTERM")
print(len(full.encode("utf-8"))) # 48
print(len(cut.encode("utf-8"))) # 39

Basically, your variable labels are too long

[ofajardo]

SAS does apparently the same when converting this sas7bdat to XPT, so the behaviour looks correct to me, here an excerpt from the SAS log

NOTE: The variable label 病史/手术史/家族史报告名称/过敏源 has been truncated to
病史/手术史/家族史报告名称/.

[guoranba]

Thanks for your quick response and detailed explanation. It seems we can only get xpt file in UTF-8 format by default using pyreadstat.pyreadstat.write_xport, right? I wonder if it's possible to add optional parameter of 'encoding' in writing to a SAS xpt file so that we can get a correct variable label when setting encoding='gbk'.

[ofajardo]

currently it is not supported, and although I am not 100% sure, I think the backend Readstat does not support it