3 posts tagged with "ci"

Question​

When using Atlas on my macOS, I can run atlas migrate apply without any issues, and it applies the migrations successfully. But when running in CI on linux, I get the following error: Error: checksum mismatch

Answer​

This issue is likely due to differences in line endings between operating systems. macOS uses LF (Line Feed) for line endings, while Windows uses CRLF (Carriage Return + Line Feed). When you run Atlas commands in a Docker container, it may be using a different line ending format than what your migrations were created with.

To resolve this issue, you can try the following steps:

1. Add a .gitattributes file to your repository with the following content:

# Ensure all text files use LF line endings
* text=auto eol=lf

2. Commit the .gitattributes file to your repository.
3. If you have existing migration files that were created with different line endings, you may need to normalize them. You can do this by running the following command in your repository:

# Remove all files from the index (but keep them on disk)
git rm --cached -r .
# Re-checkout the files from the repository
git reset --hard

How to detect schema drift in CI and ensure the migration directory is up to date with the desired schema state - ORM, HCL, SQL, or any other schema definition?

Question​

What encoding is used by Atlas for schema files? How do you avoid encoding issues with PowerShell on Windows?

Answer​

Atlas uses UTF-8 to encode schema files and migration files generated by Atlas. UTF-8 is the default encoding on nearly all system terminals, but on Windows, PowerShell uses UTF-16 by default. This can cause an encoding issue when Atlas loads the schema files generated by the inspect command.

For example, running the following command in PowerShell will create the schema.sql file with UTF-16 encoding:

atlas schema inspect -u <URL> > schema.sql

When loading this file for another command, like atlas migrate apply, this will lead to an error because Atlas expects UTF-8 encoding:

Error: schema.sql:1: Error 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1

The same problem happens with the external_schema datasource:

data "external_schema" "gorm" {
  program = [
    "go",
    "run",
    "-mod=mod",
    "ariga.io/atlas-provider-gorm",
    "load",
    "--path", "./path/to/models",
    "--dialect", "postgres", // | mysql | sqlite | sqlserver
  ]
}

You will get the following error because the output from program is in UTF-16 encoding:

Error: data.external_schema.gorm: running program C:\\Program Files\\Go\\bin\\go.exe: 13:49: illegal character U+0073 's' in escape sequence (and 6 more errors)
exit status 1

To prevent these errors, we recommend changing the default encoding of PowerShell on Windows to UTF-8 by creating a default profile.

Create or edit your PowerShell profile to set the default encoding to UTF-8. For most users, this is done at the $PROFILE location (typically $HOME\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1). This does not require administrator privileges and only affects your user account.

If you want to set the encoding for all users (requires administrator privileges), you can create or edit the profile at the $PSHOME location (typically found at C:\Windows\System32\WindowsPowerShell\v1.0). See Microsoft's documentation for more details.

Add the following content to your profile file:

$PSDefaultParameterValues['Out-File:Encoding'] = 'utf8'
$PSDefaultParameterValues['*:Encoding'] = 'utf8'

After creating the profile, restart all Terminal apps by closing and reopening them. Then, run $PSDefaultParameterValues['Out-File:Encoding'] to confirm that the default encoding has been updated:

$ $PSDefaultParameterValues['Out-File:Encoding']
utf8
$ $PSDefaultParameterValues['*:Encoding']
utf8