The Hidden Costs of Misconfiguration
by John Ryan, Varsha Munishwar — Sep 27, 2022
A Cryptic Error ¶
Take a look at this error message:
... Updating resource service/petc (serving.knative.dev/v1) API server says: admission webhook "validation.webhook.serving.knative.dev" denied the request: validation failed: "PORT" is a reserved environment variable: spec.template.spec.containers.env.name ...
What you’re looking at is the tail end of a 30-minute circuitous journey locating and collecting logs after a particular service apparently failed to deploy. 🥵
The person trying to decipher this cryptic-to-them error message wasn’t versed in the intricacies of Knative services. What they did know was that there’s a configuration option in the package they are using to inject environment variables.
Here’s the simplified schema:
#! schema.yaml #@data/values-schema --- service_name: "" additional_env_values: - name: "" value: ""
and they had supplied these data values:
# values.yaml --- service_name: hello additional_env_values: - name: PORT value: "5432"
When they deployed their application, all seemed to be in order… until it wasn’t.
Now, in this reduced example, the source of the error is probably obvious to the reader. But consider when there are dozens (if not more) Data Values involved… or if there weren’t so obvious matchable strings in the error message and inputs.
And at least in this case, there’s an admission webhook involved. Imagine if the workload simply quietly failed to run? Or ignored the invalid configuration altogether??!?
It’s a recipe for pain. 😖
Shift Left Configuration Errors ¶
The key to avoiding this kind of suffering is to “shift left” the validating input values.
By “shift left”, we mean moving a test or check to an earlier step in the process. This has two key effects:
- the end-user gets immediate feedback, saving them (potentially hours) of troubleshooting;
- the message they get is in terms of the inputs they provided, not in places where that input was used.
We’ve actually grown to expect this kind of thing from web applications we use every day. Signing up for a service, we’re often asked for our email address. Usually, we get immediate feedback when we put in a value that is not a well-formed email address.
ytt Validations ¶
ytt v0.43.0, you now have the ability to give your user that more useful/pleasant experience of reporting an
erroneous input right away.
Using our example above, the author could simply include a validation annotation:
#! schema.yaml #@data/values-schema --- service_name: "" additional_env_values: - + #@schema/validation ("environment variable name, expect PORT (which is reserved)", lambda v: v != "PORT") name: "" value: ""
- each value given for
additional_env_valueswill be validated;
name:has a validation rule defined:
- the first parameter is a user-friendly message, describing what a valid value is;
- the second parameter is an expression that implements that rule (here, the value of
name:, passed to
Instead of having to find and make heads-or-tails of the error message at the top of this article…
… as soon as
ytt is invoked, they see this:
$ ytt -f schema.yaml --data-values-file values.yaml ytt: Error: Validating final data values: additional_env_values.name from: values.yaml:4 - must be: environment variable name, expect PORT (which is reserved) (by: schema.yaml:6)
- the data value they supplied
additional_env_values.nameis directly referenced (including filename and line number);
- the definition of a valid value is given.
It’s seconds, not minutes (or hours) to learn what when wrong, how, and what they can do to fix it.
Now that’s a delightful experience. 🥳
Where to, from here? ¶
Learn more about
- See a demo of validations in action (click “show more” for a table of contents of the video).
- Upgrade to ytt v0.43.0 or later.
- Get a gentle introduction through our How To Write Validations.
- Get started quick with our Validations Cheat Sheet.
- Dive into the syntax and inventory of out-of-the-box rules in the @schema/validation reference.
Join the Carvel Community ¶
We are excited to hear from you and learn with you! Here are several ways you can get involved:
- Join Carvel’s slack channel, #carvel in Kubernetes workspace, and connect with over 1000+ Carvel users.
- Find us on GitHub. Suggest how we can improve the project, the docs, or share any other feedback.
- Attend our Community Meetings! Check out the Community page for full details on how to attend.
We look forward to hearing from you and hope you join us in building a strong packaging and distribution story for applications on Kubernetes!