-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Payloads as varchar and charset of payloads #164
Comments
@Francois-BellegardeOSF what collation is your database using? SELECT DB_NAME() As DB, DATABASEPROPERTYEX(DB_NAME(), 'Collation') AS Collation |
The schema we use assumes you are using a Support for UTF-8 collation was added in SQL Server 2019: https://techcommunity.microsoft.com/t5/sql-server-blog/introducing-utf-8-support-for-sql-server/ba-p/734928. |
When I create a DB, it still defaults to SQL_Latin1_General_CP1_CI_AS. And apparently, this is also what IT did in production. I tested reimporting the DB after editing model.xml in the .bacpac and it solved this issue. Thank you for clearing this up. Could this not have been made to explicitly use a UTF8 collation on the columns? |
@cgillum |
@Francois-BellegardeOSF yes, this particular collation was chosen for performance reasons. I also like to use it at development time to make sure we maintain compatibility with case-sensitive databases. But you can still use our schema with non-case sensitive databases. The UTF-8 issue should be the only compatibility concern. |
@cgillum |
@cgillum , we can't change the collation for our production running DB. Another the DTF tables are in the same DB with other business tables. To change collation will break our other logic. |
The choice of using varchar(max) instead of nvarchar(max) for payloads causes issues when payloads contain characters not supported by varchar.
The workaround would be to encode every message, but that is a solution more adapted for new apps, and even then, it's a bit awkward.
Migrating existing apps could really be problematic.
For example, I added an emoji to an activity output. Note the '??' that have replaced it. When using Azure Storage, there is no issue with charset.
The text was updated successfully, but these errors were encountered: