Schema Migration Guide

Schema documents provide a way to declare Data Values with their types, and default values. Without Schema, validating the presence of Data Values requires additional ytt configuration containing Starlark assertions.

• Learn more about writing Schema
• Read the detailed Data Vaues Schema Reference

How do I, a configuration author, migrate my ytt library to use Schemas? ¶

To make use of the Schema feature, your ytt invocation must first contain files using the Data Values feature. Migrating to Schemas involves converting your Data Values files into a Schema file.

Single Data Values file ¶

Starting with a single Data Values file, values.yml:

#@data/values
---
key1: myVal
key2: 8080

Convert this Data Values file into Schema by changing the top level annotation in the document to say #@data/values-schema, and (optional) rename values.yml to values-schema.yml:

#@data/values-schema
---
key1: myVal
key2: 8080

Now simply include this Schema file in your ytt invocation to receive the benefits of ytt Schemas.

Note: If your Data Values file contains arrays (ie. ["example"], - example), be sure to provide default values for arrays.

Multiple Data Values files ¶

Sometimes, it makes sense to split Data Values into multiple files. If this is your situation, there are a few things to note.

Given a ytt configuration with two Data Values files:

$ tree .
.
├── config.yml
├── values-1.yml
└── values-2.yml

values-1.yml:

#@data/values
---
key1: myVal
key2: 8080

values-2.yml:

#@data/values
---
key2: 8088
#@overlay/match missing_ok=True
key3:
  host: registry.dev.io
  port: 8080

You can convert each Data Values document into its own Schema document by following the steps to convert a single Data Values file.

Multiple Schemas combine exactly like Data Values, via overlays: the first Schema establishes the base set of “data value” declarations, and subsequent Schema files are overlays on top of that base.

values-1-schema.yml:

#@data/values-schema
---
key1: myVal
key2: 8080

values-2-schema.yml:

#@data/values-schema
---
key2: 8088
#@overlay/match missing_ok=True
key3:
  host: registry.dev.io
  port: 8080

Now just include these Schema files in your ytt invocation instead of the Data Values files.

Multiple Data Values files + Private Libraries ¶

If your configuration depends on a ytt library — outlined in the library module docs — there are a few points to note.

$ tree .
.
└── config
    ├── config.yml 
    ├── values-1.yml
    ├── values-2.yml
    └── _ytt_lib
        └── lib
            ├── service.yml
            └── values.yml

config.yml:

#@ load("@ytt:data", "data")
#@ load("@ytt:library", "library")
#@ load("@ytt:template", "template")

#@ lib = library.get("lib").with_data_values(data.values)
--- #@ template.replace(lib.eval())

Using the same values-1.yml and values-2.yml files from the multiple Data Values files Schema migration example above.

Migrating to Schema happens one library at a time. Let’s start with the root library, which includes everything at and below the file system level where the ytt invocation was called, not including the _ytt_lib folder:

.
└── config
    ├── config.yml
    ├── values-1.yml
    └── values-2.yml

As seen in the previous example, migrating this library to Schemas simply involves converting each values-1.yml and values-2.ymlinto a Schema file.

Now we have migrated the root library to use Schemas, and the ytt invocation will succeed as the same as before. Each library can independently opt-in to using Schemas.

$ tree .
.
└── config
    ├── config.yml 
    ├── values-1-schema.yml
    ├── values-2-schema.yml
    └── _ytt_lib
        └── lib
            ├── service.yml
            └── values.yml

Migrating a private library to use Schemas involves the same process as the root library. You can narrow the context to just the children of the _ytt_lib directory:

└── _ytt_lib
    └── lib
        ├── service.yml
        └── values.yml

Now simply follow the steps in either of the previous examples to migrate the private library to use Schemas.

How do I provide default values for an array? ¶

Arrays in Schemas are handled differently than other types: exactly one element is specified in the array, and that value is only used to infer the type of that array’s elements — the default value, by default, is an empty list (i.e. []).

The example below shows how to define an array in a Schema and then provide default values via the @schema/default annotation.

values-schema.yml:

#@ def default_conns():
- host: registry.dev.io
  port: 8080
  transport: tcp
#@ end

#@data/values-schema
---
#@schema/default default_conns()
key: 
- host: ""
  port: 0
  transport: ""
  insecure_disable_tls_validation: false

Given that schema, if a template file were to use the key data value:

key: #@ data.values.key

this would output

key:
- host: registry.dev.io
  port: 8080
  transport: tcp
  insecure_disable_tls_validation: false

How do I mark a section of Data Values as “optional”? ¶

Sometimes your configuration includes a section of Data Values that are not typically used or in some way optional.

If this the case, consider the guidance in Writing Schema: Marking a Data Value as Optional, use the @schema/nullalble annotation to default such a section to null.

How do I mark a Data Value as containing any kind of YAML? ¶

For those looking to relax the typing that Schema applies to Data Values, the @schema/type any=True annotation can be used to override the inferred typing on the node it annotates and its children.

Situations like this are covered in detail in Writing Schema: Specific Use-Cases