The Biztory Snowflake x Slack Connector is an innovative application designed to seamlessly integrate Slack data in your Snowflake environment.

1. What is the Snowflake x Slack Connector?

This powerful connector allows you to effortlessly access and retrieve a wide range of data from the Slack API, including users, channels, messages, and message threads. With the ability to select specific endpoints and set customizable schedules, this application ensures that your Slack data is automatically synchronized with your Snowflake database.

All data is retrieved incrementally, meaning the application keeps track of state to determine what data has already been collected,  ensuring that only new or updated information is fetched. This efficient approach minimizes redundant data retrieval, optimizing both performance and storage.

You have the flexibility to choose the name of the new database and schema in which you want to store the retrieved Slack data, enabling a fully tailored experience. The application features an intuitive configuration page where all settings can be easily managed, making setup and adjustments straightforward.

To keep you informed and in control, the Snowflake Slack Connector provides comprehensive logging capabilities. The "API Calls"-page offers real-time monitoring of all API interactions, allowing you to track the application's communication with Slack. Additionally, the "Logs"-page provides detailed insights into the applications operations, giving you a clear overview of its activities and performance.

2. Snowflake x Slack Connector: Endpoints

The Snowflake Slack Connector offers several endpoints...

2.1 Users Endpoint

The first endpoint focuses on retrieving all users within the Slack account. This includes gathering detailed information about each user, such as their profile details, activity status, and other relevant metadata. This endpoint is essential for organizations looking to analyze user activity, communication patterns, or simply maintain an up-to-date directory of all Slack users.

2.2 Channels Endpoint

The Channels endpoint is dedicated to retrieving all public channels in the Slack account. By accessing this endpoint, you can gain insights into the structure and organization of your Slack workspace, including the channels available for collaboration, their purposes, and the members associated with each channel.

2.3 Messages Endpoint

The Messages endpoint is perhaps the most versatile, allowing you to retrieve all messages from specific channels. The Snowflake Slack 
Connector offers three distinct options for selecting which channels to retrieve data from:

• Option 1 - OAuth-Only Channels: The application retrieves messages only from channels that the OAuth connection (i.e., the authenticated user) is already a member of. This option is ideal for those who want to ensure that only pre-approved channels are included in the data retrieval process.
  
  
• Option 2 - Top Two Channels: In this option, the application will automatically join the two public channels with the most members and retrieve messages from them. This option is particularly useful for testing the application and ensuring it is functioning correctly in a live environment, without 
  overwhelming the system with data from too many channels.
  
  
• Option 3 - All Public Channels: For those who need comprehensive data, this option allows the application to join and retrieve messages from all public channels automatically. This ensures that no data is missed and is ideal for large-scale analysis across the entire Slack workspace.

2.4 Threads Endpoint

The Threads endpoint is designed to retrieve all thread messages associated with specific parent messages in your Slack channels. This feature is particularly useful for capturing in-depth discussions that occur within threads, ensuring that all contextual information is preserved and accessible in your Snowflake environment.

**Important**

Enabling the Threads endpoint can significantly impact the performance of the application. This is because retrieving thread messages requires a separate API call for each parent message that has associated threads. For example, if there are 20,000 messages with thread responses, the application will need to make 20,000 individual API calls to retrieve all the associated thread messages.

To manage this load, the application is designed to make a maximum of 1000 calls to the Threads endpoint per cycle. While this approach helps prevent overwhelming the Slack API and maintains the application's stability, it also means that retrieving all thread messages can take a considerable amount of time, especially in larger workspaces with extensive use of threaded conversations.

You should carefully consider your need for thread data against the potential increase in processing time when enabling this endpoint. The Snowflake Slack Connector provides this functionality with performance in mind, but depending on the volume of threaded messages, patience may be required to capture all the desired data.


3. How does the Snowflake x Slack Connector work?

The Snowflake Slack Connector is designed with user simplicity and control in mind, with everything managed through a single, intuitive configuration page. This page serves as the central hub for configuring and managing how the application interacts with your Slack data.

3.1 Configuration Settings

At the top of the Config Page, you can set the destination database and schema where the retrieved Slack data will be stored. The application will automatically create them, ensuring a seamless setup process.

**Currently it's not possible to use an exsiting database for the output**.

Once the destination is configured, you can proceed to select the desired endpoints (Users, Channels, Messages, Threads) and choose a suitable schedule for data retrieval (e.g., daily, hourly, every 5 minutes).

3.2 Starting Ingestion

After configuring the endpoints and schedule, you can start the data ingestion process by pressing the "Start Ingestion"-button.

This action schedules the corresponding Snowflake task, enabling the selected endpoints and initiating the data retrieval process based on the configured schedule. Once the ingestion starts, the configuration details are saved in the "Existing Config"-section on the configuration page, making it easy to review and manage settings.

3.3 Updating Configuration

Flexibility is a key feature of the Snowflake Slack Connector. You can easily overwrite the existing configuration by selecting new endpoints, adjusting schedules, or changing the destination database and schema. To update the config, simply make the desired changes on the Configuration page and press the "Start Ingestion"-button again. If a Snowflake task is already running under a previous configuration, it will complete its current cycle before applying the new settings on the next scheduled trigger.

This approach ensures that the application remains responsive and adaptable, allowing you to fine-tune the data retrieval process without interruption.


4. Snowflake x Slack Connector: Schedules

The Snowflake Slack Connector offers flexible scheduling options to accommodate various data retrieval needs. 

You can choose from the following schedules:

• Once Every Day
• Once Every Hour
• Every 3 Hours
• Every 5 Minutes

These scheduling options are available for all endpoints, allowing you to set up the application according to your data synchronization preferences. Whether you need a daily update of your Slack data or more frequent synchronization throughout the day, these options provide the flexibility to meet your specific requirements.

It's important to remember that for each cycle, the application retrieves data for every selected endpoint. If you enable multiple endpoints, the application will sequentially process each one during the scheduled run. While all schedules work with every endpoint, it's crucial to consider the impact on performance, especially when dealing with the Threads endpoint.

While the application will handle this load, choosing a very short interval, such as every 5 minutes, along with selecting all endpoints, including Threads, may result in prolonged processing times. You should carefully consider the frequency of your schedule and the potential impact on performance, especially when enabling the Threads endpoint, to make an informed decision that balances data needs with application efficiency.

 

5. Snowflake x Slack Connector: Security and Connection Setup

All the actions required to securely connect and configure the Snowflake Slack Connector are managed through the Security Page of the application. This page is easily accessible next to the Manage Access button located in the top right corner of the interface. Once inside the Security Page, you should navigate to the Connection tab to set up and manage the Slack integration."""

User Actions: Key steps and considerations

1. Connecting to Slack
The Snowflake Slack Connector connects to Slack via a dedicated Slack application. More detailed information about Slack applications can be found in the official Slack documentation. You can create a new Slack application by following this link.

It's crucial to understand that this Slack application will be used for the OAuth connection with Slack. For now, it's important to ensure that the OAuth token obtained **does not expire**, as token expiration handling will be addressed in a future update of the Snowflake Slack Connector. Once the Slack application is created, you can add the Client ID and Client Secret in the OAuth section within the security settings of the Snowflake Slack Connector application.

2. Token Security and OAuth Behavior
The OAuth token retrieved during the connection process is securely stored within a Snowflake secret, ensuring that your credentials remain safe and protected. However, you should be aware of a known issue where the OAuth popup might get stuck in an infinite loop after the connection is made. Although this is something we are actively working on, closing the popup manually after the connection is established will still work as expected, and the application will be fully functional.

3. External Access Integration
To ensure the Snowflake Slack Connector can successfully communicate with Slack, you must configure the External Access Integration within the application. This integration allows the Snowflake environment to securely connect to Slacks API, specifically limiting the connection to the slack.com domain. This restriction enhances security by ensuring that data exchanges are only conducted with Slack, preventing unauthorized access to other domains.

By following these steps in the Security Page, you can confidently and securely establish the necessary connections for the Snowflake Slack Connector, enabling seamless data integration with their Slack environment.