SQL Server provides a powerful feature called templates that can greatly simplify the process of creating new pipelines with specific functionality. Templates allow you to quickly create a workflow without the need to spend time designing and developing the pipeline from scratch.

Why Use Templates?

SQL Server templates are predefined pipelines that are available in the Template Gallery. These templates can be used to create pipelines for various tasks such as data copying, data transformation, and more. By using templates, you can save time and effort when you need to perform repeated provisioning for any resource.

How to Use Templates

To create a new pipeline from an existing template, open SQL Server and navigate to the Azure Data Factory using the Azure Portal. From the Author and Monitor main page, click on the “Create Pipeline from Template” option. This will take you to the Template Gallery page where you can browse and select from a variety of predefined pipeline templates.

Each template in the gallery provides a description, tags, and services that can be used as a source or a sink for the pipeline. You can also preview the pipeline template design to see the activities that are used to achieve the desired functionality.

If you decide to use a template, you will need to select or create a Linked Service for each source and sink required in the selected template. You can then validate the connection properties for each Linked Service using the Test Connection option.

Once you have configured the necessary Linked Services, you have two options. You can either export the pipeline JSON file to be saved on your machine, or you can use the template to create a new pipeline in your Azure Data Factory.

If you choose to create a new pipeline, you will be notified of the number of resources that will be created from the selected template. The created pipeline will then be opened in the pipeline Author window, allowing you to customize it based on your requirements.

Creating Custom Templates

In addition to using predefined templates, SQL Server also allows you to save your existing pipelines as templates. This can be useful when you want to reuse a pipeline with different inputs or by different teams.

To save an existing pipeline as a template, open the pipeline in the Author window and click on the “Save as template” option. Provide a unique name, description, and specify the GIT location where the template will be saved. You can also add tags and services to make it easier to search for the template.

The saved custom pipeline template will be visible to all Data Factory users with access to the GIT repository. It can be edited, exported, and used by different users to create similar pipelines without starting from scratch.

Conclusion

SQL Server’s pipeline templates make it easy to create new pipelines with just a few clicks. By using templates, you can save time and effort in designing and developing pipelines from scratch. Additionally, you can save your existing pipelines as templates to be used by other teams or in different development stages. Templates are a powerful feature that can greatly enhance your productivity in SQL Server.

Stay tuned for more articles on SQL Server and Azure Data Factory!