tmo Labs

#003 Want A peace time With Redshift ?

August 7, 2024

Right-size the Redshift Cluster: Ensure that your cluster’s size matches your workload requirements to optimise performance and cost.

Enable Auto WLM (Workload Management): Automate workload management to efficiently allocate resources based on query demands.

Segregate the WLM Queue: Organise queues based on workload type and priority for better resource allocation.

Create Groups Based on Workload Type: Group similar workloads together to streamline resource management.

Attach Groups to the Respective Queue: Assign each group to the appropriate WLM queue to ensure resources are allocated efficiently.

Segregate Users and Attach to Respective Groups: Ensure users are assigned to the correct groups to maintain consistent workload management.

Enforce Timeouts in the Cluster Parameter Group: Set query timeouts to prevent runaway queries from consuming resources.

Monitor Long-Running Queries: Keep an eye on long-running queries and optimise them to improve performance.

Implement a Manual Vacuuming Strategy: Regularly vacuum your tables to reclaim space and maintain performance.

Perform Regular Table Archival: Archive old data periodically to keep your tables lean and efficient.

Enable Auto Concurrency Scaling: Automatically scale resources to handle increased concurrency without impacting performance.

Monitor Concurrency Scaling Operations: If you notice frequent scaling throughout the day, consider increasing the number of nodes.

Identify the Right Distribution Key and Sort Key: Choose appropriate distribution and sort keys to optimise data distribution and query performance.

Adopt the Right Distribution Style: Select the distribution style that best suits your data and query patterns.

Address Distribution Key and Sort Key Skew: Fix any skew in your distribution and sort keys to balance data distribution across nodes.

Monitor Waiting Queries in WLM Queue: Identify queries that are waiting in the WLM queue and adjust resources or priorities as needed.

Choose the Right Merge Strategy: Optimise merge operations to minimise impact on performance.

Avoid Frequent Scans of Large Data with Spectrum: Minimise the use of Amazon Redshift Spectrum for heavy data scans to reduce costs and improve performance.

Monitor Costs: Regularly review your Redshift usage and costs to ensure you’re staying within budget.