# Scaffolder advanced usage This article describes advanced features you can use with a Scaffolder block in a [Workflow](/streamline/workflows.md): Customizing the autogenerated `cortex.yaml`, building complex templates that use inputs collected by Cortex, building templates that add to existing services, and more. ### Repository cloning behavior When running a Cookiecutter template, Cortex performs a shallow clone of the template's source repository. A shallow clone fetches the entire repository tree, but only at the current `HEAD` commit— not the full history. Behavior on the target side depends on the run type. When a Scaffolder run creates a new repository, Cortex provisions it directly—no clone of an existing target is involved. When a run opens a pull request against an existing repository, Cortex clones that target repository with a shallow clone of the entire repository. Sparse checkout on the target can be used when the run produces only additive changes, i.e. new files only, with no updates or deletions to existing files, since only the paths being added need to be materialized. ### Templates that add to existing services A powerful workflow that the Scaffolder unlocks is the ability to templatize code to be added to *existing* services – for example, you may want templates that developers can use to add Terraform modules, CI pipelines, or build/deploy scripts to a service that's already been built. To do so, you'll want to create a template that generates the file or directory you'd like to add to your existing service, and enable the PR-only toggle when adding the template to your workspace. Keep in mind that your template *always* needs to generate a directory as expected by cookiecutter. {% hint style="info" %} The Scaffolder merges files with a recursive copy. If your template generates files or directories that already exist in the target repo, Cortex will *not* attempt to merge the content of the files. Instead, we do a recursive copy, replacing any files that exist in the target. {% endhint %} #### Root-level files If you're trying to add files to the root of an existing service, your template should generate the files at the root of the generated directory. Keep in mind that you can use template variables for the filenames as well. The expected file structure of your template would be: ``` (template-folder)$ ls -l cookiecutter.json {}/ {}-generated-file.js my-file-to-generate.yml ``` Note that the files live under a nested, templated directory. When we open a PR, the PR will still only contain the two nested files at the *root of the PR*. Essentially, the PR will be: ``` {}-generated-file.js my-file-to-generate.yml ``` #### Directories If you're looking to generate a directory, the template should *generate* the desired directory as the output. Although `cookiecutter` by default generates the output of the entire template into a directory, the PR feature copies output of the template out of the generated directory into the root (recursively). For example, if you want your template to generate the directory `infra-files-{}` (or generate a file into this directory), your structure should look like: ``` (template folder)$ ls -l cookiecutter.json {}/ infra-files-{}/ my-generated-file.json ``` Note that you can use templated variables when generating the directory name or files in the directory. ### Overriding the autogenerated Service Descriptor By default, the Scaffolder generates an [entity descriptor file](/ingesting-data-into-cortex/entities-overview/entities.md#defining-entities-via-yaml-file) named `cortex.yaml` in the root of the repository when used to create a service. This file contains some basic information which the user inputs during the flow such as [entity ownership](/ingesting-data-into-cortex/entities-overview/entities/ownership.md), [groups](/ingesting-data-into-cortex/entities-overview/entities/groups.md), and [external docs](/ingesting-data-into-cortex/entities-overview/entities/external-docs.md). In some cases, you may want to generate your own Service Descriptor. This may because you want to add [custom data](/ingesting-data-into-cortex/entities-overview/entities/custom-data.md), auto-generate groups based on the template they've selected, add placeholder links, or other such advanced use cases. To override the Service Descriptor generated by Cortex, just include a `cortex.yaml` or `cortex.yml` file in the template. If Cortex detects that the template has generated either one of these files, the Scaffolder will skip the Service Descriptor generation step. ### Hiding template variables in the UI As the complexity of your template grows, you may have variables in your `cookiecutter.json` that you want to hide from the UI, for example if they're autogenerated using Python or they're not something you want users to override at all. To hide fields, just add a `@cortex_hidden_keys` field containing a list of variables you want to hide. ```json { "project_name": "My Project", "project_slug": "{}", "@cortex_hidden_keys": ["project_slug"], "version": "0.0.1" } ``` We hide the `version` key by default, so you don't need to add that to the list! If you have "private variables" (variables starting with `_`, such as `"_my_secret_variable"`), they will automatically be hidden from the UI. ### Consuming Cortex fields in a template The Cortex Scaffolder flow requires users to input additional information aside from the fields defined in the `cookiecutter.json` file when creating a service. This includes: * Git repository details * Ownership * [Cortex tag](/ingesting-data-into-cortex/entities-overview/entities.md#cortex-tag) * [Groups](/ingesting-data-into-cortex/entities-overview/entities/groups.md) * Dependencies * Template details You may want your template to use this information, without duplicating these fields in your `cookiecutter.json` and having developers fill out the information twice! For example, you may want to use the name of the git repo that's created while generating a deployment script in the template. These fields are available to your template in a magic `_@cortex_inputs` variable. #### The `_@cortex_inputs` variable The structure of `_@cortex_inputs` is a dictionary that will always contain the following: ```json { "git": , // varies based on git provider, see below "user": , "template_details": } ``` See below for details on the `git` object, which contains information on the repository that Cortex is creating for the new project. {% hint style="info" %} You can access this variable in your template the same way you do for any of your own variables: `{}`. {% endhint %} If your template has the **"Requires a new service"** flag enabled, `_@cortex_inputs` will additionally contain the following: ```json { "service_details": { "tag": "my-service", "name": "My Service", "description": "Optional description if defined", "owners": [ , , ], "serviceGroups": ["list", "of", "groups"], "dependencies": [ , , ] } } ``` **Git details** The git object shown above is different based on the provider you're using. The structure is shown below. **GitHub** ```json { "git": { "repository": } } ``` **GitLab** ```json { "git": { "repository": } } ``` **Bitbucket** ```json { "git": { "repository": } } ``` **Azure DevOps** ```json { "git": { "repository": } } ``` ### Extending Cookiecutter: Cortex field configurations You can add metadata to each variable that Cortex will display during the scaffolding process. #### Adding descriptions and placeholders to inputs To help guide users during scaffolding, you can configure input fields with: * a description (appears beneath the input field) * a placeholder (appears inside the input box before the user types anything) These are defined in the `_cortex_field_configurations` object in your `cookiecutter.json`. Here's an example `cookiecutter.json`, where `full_name` is an input consumed by the template. We've defined `_cortex_field_configurations` to add a description and a placeholder to the UI field for `full_name`. ```json { "full_name": "Albert Einstein", "_cortex_field_configurations": { "full_name": { "description": "Enter the full name of the new team member", "placeholder": "e.g., Jane Doe" } } } ``` {% hint style="info" %} The `use_default` key is no longer supported. While default values (like `"Albert Einstein"`) can still be defined in your template, they will not be surfaced in the UI unless manually entered by the user. {% endhint %} --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.cortex.io/streamline/workflows/scaffolder/advanced-usage.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.