Carvel Logo

Schema Migration Guide

Schema documents provide a way to declare Data Values with their types, and default values. Without Schema, validating the presence of Data Values requires additional ytt configuration containing Starlark assertions.

How do I, a configuration author, migrate my ytt library to use Schemas?

To make use of the Schema feature, your ytt invocation must first contain files using the Data Values feature. Migrating to Schemas involves converting your Data Values files into a Schema file.

Single Data Values file

Starting with a single Data Values file, values.yml:

#@data/values
---
key1: myVal
key2: 8080

Convert this Data Values file into Schema by changing the top level annotation in the document to say #@data/values-schema, and (optional) rename values.yml to values-schema.yml:

#@data/values-schema
---
key1: myVal
key2: 8080

Now simply include this Schema file in your ytt invocation to receive the benefits of ytt Schemas.

Note: If your Data Values file contains arrays (ie. ["example"], - example), be sure to provide default values for arrays.

Multiple Data Values files

Sometimes, it makes sense to split Data Values into multiple files. If this is your situation, there are a few things to note.

Given a ytt configuration with two Data Values files:

$ tree .
.
├── config.yml
├── values-1.yml
└── values-2.yml

values-1.yml:

#@data/values
---
key1: myVal
key2: 8080

values-2.yml:

#@data/values
---
key2: 8088
#@overlay/match missing_ok=True
key3:
  host: registry.dev.io
  port: 8080

You can convert each Data Values document into its own Schema document by following the steps to convert a single Data Values file.

Multiple Schemas combine exactly like Data Values, via overlays: the first Schema establishes the base set of “data value” declarations, and subsequent Schema files are overlays on top of that base.

values-1-schema.yml:

#@data/values-schema
---
key1: myVal
key2: 8080

values-2-schema.yml:

#@data/values-schema
---
key2: 8088
#@overlay/match missing_ok=True
key3:
  host: registry.dev.io
  port: 8080

Now just include these Schema files in your ytt invocation instead of the Data Values files.

Multiple Data Values files + Private Libraries

If your configuration depends on a ytt library — outlined in the library module docs — there are a few points to note.

$ tree .
.
└── config
    ├── config.yml 
    ├── values-1.yml
    ├── values-2.yml
    └── _ytt_lib
        └── lib
            ├── service.yml
            └── values.yml

config.yml:

#@ load("@ytt:data", "data")
#@ load("@ytt:library", "library")
#@ load("@ytt:template", "template")

#@ lib = library.get("lib").with_data_values(data.values)
--- #@ template.replace(lib.eval())

Using the same values-1.yml and values-2.yml files from the multiple Data Values files Schema migration example above.

Migrating to Schema happens one library at a time. Let’s start with the root library, which includes everything at and below the file system level where the ytt invocation was called, not including the _ytt_lib folder:

.
└── config
    ├── config.yml
    ├── values-1.yml
    └── values-2.yml

As seen in the previous example, migrating this library to Schemas simply involves converting each values-1.yml and values-2.ymlinto a Schema file.

Now we have migrated the root library to use Schemas, and the ytt invocation will succeed as the same as before. Each library can independently opt-in to using Schemas.

$ tree .
.
└── config
    ├── config.yml 
    ├── values-1-schema.yml
    ├── values-2-schema.yml
    └── _ytt_lib
        └── lib
            ├── service.yml
            └── values.yml

Migrating a private library to use Schemas involves the same process as the root library. You can narrow the context to just the children of the _ytt_lib directory:

└── _ytt_lib
    └── lib
        ├── service.yml
        └── values.yml

Now simply follow the steps in either of the previous examples to migrate the private library to use Schemas.


How do I provide default values for an array?

Arrays in Schemas are handled differently than other types: exactly one element is specified in the array, and that value is only used to infer the type of that array’s elements — the default value, by default, is an empty list (i.e. []).

The example below shows how to define an array in a Schema and then provide default values via the @schema/default annotation.

values-schema.yml:

#@ def default_conns():
- host: registry.dev.io
  port: 8080
  transport: tcp
#@ end

#@data/values-schema
---
#@schema/default default_conns()
key: 
- host: ""
  port: 0
  transport: ""
  insecure_disable_tls_validation: false

Given that schema, if a template file were to use the key data value:

key: #@ data.values.key

this would output

key:
- host: registry.dev.io
  port: 8080
  transport: tcp
  insecure_disable_tls_validation: false

How do I mark a section of Data Values as “optional”?

Sometimes your configuration includes a section of Data Values that are not typically used or in some way optional.

If this the case, consider the guidance in Writing Schema: Marking a Data Value as Optional, use the @schema/nullalble annotation to default such a section to null.

How do I mark a Data Value as containing any kind of YAML?

For those looking to relax the typing that Schema applies to Data Values, the @schema/type any=True annotation can be used to override the inferred typing on the node it annotates and its children.

Situations like this are covered in detail in Writing Schema: Specific Use-Cases


(Help improve our docs: edit this page on GitHub)