Revolutionize Your Data Workflow: Keboola CLI Meets Cursor AI

We at Keboola believe that there can’t be too much efficiency in data engineering. Any data professional would surely agree, there is no such thing as good enough. And while better tools sometimes offer incremental steps, pushing the needle a little step at a time, a combination of tools can lead to impressive synergies. Keboola CLI and Cursor AI are exactly that—a combination that’s more than a sum of its parts.

Read on to see how this dynamic duo can transform your workflow, making your experience in managing and developing data pipelines better than ever. 

The Ultimate AI-Powered Data Pipeline for Modern Developers

Keboola Command Line Interface (CLI) is a command line interface for managing cloud data pipelines in Windows, macOS, and Linux environments. On its own, it’s a robust tool with many use cases, but together with enhanced coding capabilities provided by Cursor AI, they create The Ultimate AI-Powered Data Pipeline for Modern Developers, if we may say so ourselves. 

Key Benefits of the Keboola CLI + Cursor Workflow

1. Seamless Integration of Data and AI: Cursor's AI understands your entire codebase, including Keboola CLI-managed pipelines, providing contextually relevant code suggestions
2. Accelerated Development Cycles: Rapidly prototype data pipelines and leverage AI for code optimization
3. Enhanced Collaboration: Share project context easily and generate AI-assisted documentation
4. Unmatched Flexibility: Develop custom connectors and adapt AI assistance to your evolving needs

Common Use Cases

• Pull your entire project to a local directory in seconds (See the init and pull commands)
• Bulk edit component configurations in your IDE
• Compare the local version with the current project state. See the diff command
• Copy a configuration as a directory in the project and between projects. See the persist command
• Apply all changes back to the project in a moment. See the push command
• Manage project history in a git repository
• Automate the whole process in a CI/CD pipeline. See GitHub Integration
• Merge and rebase Keboola Branches via Git. Learn more in the Example Use Cases section
• Distribute a single project definition into multiple projects. See the Example Use Cases section
• Multi-stage (and multi-project) environment management via Git. See the Example Use Cases section
• Locally develop and test your dbt transformation code

Getting Started with Keboola CLI and Cursor

Let's walk through setting up and using this powerful combination.

Step 1: Installation and Setup

1. Install Keboola CLI using your preferred package manager.
2. Download and install Cursor from cursor.so.
3. Open Cursor and create a new project directory for your Keboola work.

Step 2: Initializing Your Keboola Project‍

1. Create a new directory for your project and navigate to that directory by running the following:

mkdir my-keboola-project

cd my-keboola-project

Replace “my-keboola-project” with your project name or whatever you choose to name your directory.

2. Initialize project in Cursor using Keboola CLI, running kbc init, and answer different questions about your environment and project:

Keboola Storage API Host: Navigate to your project dashboard in Keboola. Copy and paste the first portion of the URL into the terminal.

Keboola Storage API Token: In your Keboola project, click your profile icon in the top right corner, select Project Settings, then select API Tokens. Here you can create a new token or use an existing one. Click here for more on the Keboola Storage API.

Version Control: Answer questions about project branches, GitHub workflows, etc.

After answering all the questions, Keboola CLI will pull your project to your local environment to start working on using Cursor.

3. Pull your Keboola configurations:

kbc sync pull

‍
Practical Example: Modifying a Keboola Configuration with Cursor AI

KBC Toy Co. is a fictional demo company that uses a Shopify storefront to sell toys and uses Keboola to manage its inventory and data. To demonstrate the power of using Cursor with Keboola, we are data engineers at KBC Toy Co. looking to improve our data operations and analytics using Cursor AI.

Let's walk through the process:

Initializing Project

Click here to see how to initialize your Keboola project in your local environment using Keboola CLI.

Understanding our Project Using Cursor AI

After we set up our project on our local environment in Cursor

1. Add relevant documents to Cursor AI’s context to improve the accuracy of responses

To help Cursor AI interact better with Keboola and more effectively assist in modifying our project, we can add the Keboola CLI, Keboola Developer Docs, and Keboola User Docs to its context.

‍Click here to see how to perform this.

2. Ask Cursor AI for an overview of our project

Press ctrl + L to open the Cursor AI window on the right-hand side of your screen. Here you can see the different tools available:
‍Cursor Chat: Use to ask questions about certain files or lines in your codebase.
‍Cursor Composer: Use to modify files in your codebase or make multifile changes at once.
‍Bug Finder: Compares uncommitted changes to the main branch and checks for errors.

We can ask Cursor AI for an overview using Cursor Chat and provide our entire codebase using @Codebase:

3. Ask Cursor for ways we can add to or improve our project.

Now that we have some ideas, let's make the changes.

‍
‍Creating a New Configuration

‍Let’s say we like one of the ideas Cursor came up with for a new transformation: Anomaly Detection to help us detect unusual patterns in orders and customer behavior for our toy store. There are multiple ways we can have Cursor help us make the necessary additions and write code, but for this example, we are going to use Cursor Composer.

Navigate to the Composer tab in our Cursor AI pane.

Now let’s ask Composer to create the new transformation for us:

Instead of just suggesting changes we can make to our project, the Composer feature can actually do it for us. 

In the AI pane on the right, we can see an explanation or overview of the changes that Composer wants to make. In the code interface, Cursor displays the changes for us to review in a similar style to reviewing a pull request or a Git diff in GitHub/VS Code.

We can either go file-by-file and accept the changes by pressing Ctrl + Shift + Y, or select Accept All if we are satisfied with all proposed changes.

The files for the new configuration are now added to your codebase! If you aren’t satisfied with the changes after you accept them, you can always revert them by clicking restore (more info).

‍Modifying Existing Configurations

‍In addition to creating new configurations, we can also update existing ones to improve our project. One option is to use Composer to make the update for us. Let’s change the date range for our Shopify Data Extractor to pull from:

Cursor will open the file and display the proposed change.

Another option is to manually identify sections of a specific file that we want to change and use Cursor to help us change them. This can be helpful for tasks like quickly rewriting functions.

To perform this, highlight the section of code you want to modify and press ctrl + K on your keyboard. A chat window will pop up where you can ask Cursor to make changes to it or ask a question about the section.

Let’s ask Cursor to modify one of our functions to work on a copy of a dataframe instead of the original:

Once we select Submit Edit, Cursor will display the changes, which you can accept or reject:

‍Push Changes Back to your Keboola Project

‍Once satisfied with the changes you have made, use Keboola CLI to push the changes back to your project:

‍

kbc push

‍
Cursor will confirm the successful push and provide a summary of the updates.

Congrats! Your changes are now successfully implemented in your Keboola project.

Tips and Tricks

Adding Appropriate Documents to Improve Cursor AI

Uploading documents for Cursor AI to reference when helping to write and change code can make Cursor AI more effective for your specific use case. 

To add docs to Cursor from a web address:

1. Navigate to Settings(Gear Icon), select Features, and scroll down to the Docs section.

2. Click + Add new doc

3. Copy and paste the doc’s URL and confirm the addition.

‍

Here are some important documents that can help Cursor AI interact with Keboola:

Keboola User Docs

‍Keboola Developer Docs

‍Keboola CLI Docs

‍To add downloaded files or local documentation to Cursor:

1. Create a new folder inside your project folder and add your files to the folder

‍

2. Open your project folder in Cursor and open the Cursor AI Pane (Ctrl + L).

3. Navigate to Settings (Gear Icon), select Features, and scroll down to the Codebase Indexing section.

4. Click on Compute Index or Resync Index if you already have an index.

‍

Now you can type @Codebase in the chat or use Ctrl + Enter to retrieve the most relevant parts of your codebase for your question, and Cursor will use the files you uploaded when generating responses. Another way to add individual files to a specific chat is by using the @files or @docs tag.

When using Cursor AI or any LLM to help manage your Keboola projects and workflows, you can add Keboola’s llms.txt file to the model’s context to significantly improve its ability to interact with Keboola.Here are some other examples of more use case-specific docs that you can add:

• Project Specifications & Data Flow Docs:  A high-level overview of how your Keboola project is structured

• Custom Component Documentation: If you’re building custom components (e.g., Python-based extractors), adding their readme files and API docs will improve code suggestions.

‍Adding Rules for Cursor AI to Follow

‍Cursor allows you to add rules that its model must follow when generating responses.

‍Global Rules

‍Global rules will be applied across all projects you open in Cursor, and can be added by modifying the Rules for AI section.

Navigate to Settings > General > Rules for AI.

This custom instruction will be included for features such as Cursor Chat and Ctrl + K.Project RulesTo add rules that only apply to your specific project, scroll slightly down from the global rules section. Project rules are stored in the .cursor/rules directory and provide granular control over AI behavior in different parts of your project.

You can also create a new rule with Ctrl + Shift + P and selecting New Cursor Rule.

‍Example use cases:

• Framework-specific rules for certain file types (e.g., SolidJS preferences for .tsx files)
• Special handling for auto-generated files (e.g., .proto files)
• Custom UI development patterns
• Code style and architecture preferences for specific folders

Utilizing @ Tags in Cursor Chat

‍Cursor offers many different @ symbol tags that are available for use within Cursor chats. These are quick and easy ways to add context to specific chats. This is useful when you want Cursor to only consider specific information for a given chat, while filtering out unnecessary information that could overcomplicate the response. Below is a description of how to use the most important ones:

‍@web - By adding the @web tag to your Cursor chat, 

You can also directly reference specific links for Cursor to search by typing @https://your-link.com and selecting

‍

‍@git - In Cursor Chat (only), you can use @Git to add git commits, diffs, or pull requests to your prompt. It may ask you to sign into your GitHub account if you haven’t already done so.

One common use case for @Git is to allow Cursor’s AI to scan the diff and look for bugs or issues that could be caused by the diff.

‍Click here to see information about all @ symbol tags available in Cursor

‍Revert Cursor Composer Changes

‍After using Composer to generate new files or change code in your codebase, you can easily revert your changes by scrolling above your message, and clicking Restore.

‍Advanced Features

Intelligent Data Operations

‍Leverage Cursor's AI to optimize your Keboola workflows:

/cursor Analyze my Keboola project and suggest optimizations for my data transformations

Cursor will review your entire project, offering insights on performance improvements, code refactoring, and best practices.

‍Automated Documentation

‍Generate comprehensive documentation for your Keboola project:

/cursor Create documentation for my Keboola project structure and data flows

Cursor will analyze your configurations and generate detailed markdown documentation, including diagrams of your data flows.

‍Conclusion: Empowering Data Teams

‍Some tools are a step forward. But the integration of Keboola CLI and Cursor AI means growth by leaps and bounds. Developing and managing data pipelines have never been easier and more accurate. Combining Keboola’s data handling and Cursor AI intelligent coding data teams can: 

• Optimize workflows and automate routine tasks
• Enhance collaboration through shared context and AI-generated documentation
• Adapt quickly to changing data needs with flexible, AI-assisted development
  
  

Start revolutionizing your data workflow today with Keboola CLI and Cursor AI— try our Free Plan.