Table of ContentsHigh-Performance Rate Limiting (Sliding Window Log)Reliable Distributed Locking & Safe ReleaseA Lightweight, High-Throughput Message Broker (Streams)Scaling WebSockets and TCP Sockets via Sharded Pub/SubThe Modern Twist: Offloading GPU/AI PipelinesProduction-Grade InfrastructureMost junior developers look at Redis and see a basic key-value store for caching database queries. When you start building high-throughput, distributed systems, you realize Redis is actually the Swiss Army knife of backend architecture.In my journey as a senior software engineer, Redis has saved my infrastructure budget and rescued my application performance more times than I can count. Here is how I actually use it in production, along with architectural patterns, code implementations, and DevOps configurations. High-Performance Rate Limiting (Sliding Window Log)API abuse can crash your services. Relying on your primary relational database to track request counts creates massive write bottlenecks. While basic tutorials show the INCR command (Fixed Window), that approach suffers from boundary bursts.In production, I implement a Sliding Window Log using Redis Sorted Sets (ZSET). This tracks the exact timestamp of every request, ensuring an accurate rate limit window. The Implementation (Python)import timeimport redisclient = redis.Redis(host='localhost', port=6379, decode_responses=True)def is_rate_limited(user_id: str, limit: int, window_seconds: int) -> bool:now = time.time()key = f"rate_limit:{user_id}"clear_before = now - window_seconds # Use a pipeline to ensure atomic execution and minimize round-trips pipe = client.pipeline() # Remove elements older than the current window pipe.zremrangebyscore(key, 0, clear_before)# Add the current timestamp as both score and member pipe.zadd(key, {str(now): now})# Get total requests in the current window pipe.zcard(key)# Set a TTL on the set to clean up idle users pipe.expire(key, window_seconds) # Execute transaction _, _, current_requests, _ = pipe.execute() return current_requests > limitSenior Tip: Atomic Lua OffloadingIf you are running this at a massive scale (tens of thousands of requests per second), move this entire logic into a Lua script. Running it inside a Lua script ensures atomicity at the Redis engine level and eliminates the network overhead of the pipeline. Here is the exact production Lua script equivalent:local key = KEYS[1]local now = tonumber(ARGV[1])local window = tonumber(ARGV[2])local limit = tonumber(ARGV[3])local clear_before = now - windowredis.call('zremrangebyscore', key, 0, clear_before)local current_requests = redis.call('zcard', key)if current_requests < limit thenredis.call('zadd', key, now, now)redis.call('expire', key, window)return 0 -- Not rate limitedelsereturn 1 -- Rate limitedendReliable Distributed Locking & Safe ReleaseIn microservices, multiple instances of a service often try to process the same data simultaneously, creating data corruption through race conditions.My Project ExperienceI once engineered a high-frequency financial ledger system. We could not allow two background workers to process the same user payout at the same second.The SolutionWe acquired a distributed lock using SET key value NX PX milliseconds. However, the critical flaw most developers make is releasing the lock blindly with DEL. If instance A takes too long, its lock expires automatically. Instance B acquires it. If Instance A finishes and blindly calls DEL, it destroys Instance B’s lock.To release a lock safely, you must use a unique token and a Lua script to ensure you only delete the lock if you own it. Safe Release Implementationimport uuiddef acquire_lock(lock_name: str, acquire_timeout: int = 10, lock_timeout: int = 30000):identifier = str(uuid.uuid4())lock_key = f"lock:{lock_name}" # NX: Set if not exists, PX: Expiry in milliseconds if client.set(lock_key, identifier, nx=True, px=lock_timeout):return identifierreturn Nonedef release_lock(lock_name: str, identifier: str) -> bool:lock_key = f"lock:{lock_name}" # Lua script ensures atomicity: check value, delete if matches lua_release = """ if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end """ result = client.eval(lua_release, 1, lock_key, identifier)return bool(result)A Lightweight, High-Throughput Message Broker (Streams)You do not always need to spin up a massive Kafka or RabbitMQ cluster. Setting those up adds massive operational complexity and infrastructure overhead.If your application needs a reliable, append-only log with Consumer Groups, at-least-once delivery guarantees, and message persistence, Redis Streams (XADD, XREADGROUP) is incredibly powerful. Production Pattern: Order Processing Ingestion# Producer: Ingesting an order eventdef ingest_order(order_id: str, user_id: str, amount: float):event_data = {"order_id": order_id,"user_id": user_id,"amount": amount,"status": "pending"}# '*' automatically generates a unique time-based ID (e.g., 1711977324-0) # maxlen=100000 prevents the stream from growing indefinitely (eviction capping) client.xadd("stream:orders", event_data, maxlen=100000, approximate=True)# Consumer: Worker processing the stream via a Consumer Groupdef process_orders(worker_name: str, group_name: str):# Setup consumer group if it doesn't exist try:client.xgroup_create("stream:orders", group_name, id="0", mkstream=True)except redis.exceptions.ResponseError:pass # Group already existswhile True:# '>' means read new messages that haven't been delivered to other consumers messages = client.xreadgroup(group_name, worker_name, {"stream:orders": ">"}, count=10, block=2000) for stream, payload in messages:for message_id, data in payload:try:# Execute heavy business logic here print(f"Worker {worker_name} processing order: {data['order_id']}") # Acknowledge successful processing (removes from PEL - Pending Entry List) client.xack("stream:orders", group_name, message_id)except Exception as e:print(f"Failed processing message {message_id}: {e}")# Leave it in PEL for retry workers to pick up via XAUTOCLAIMScaling WebSockets and TCP Sockets via Sharded Pub/SubWhen scaling real-time applications (chat, live dashboards, notification engines) to hundreds of thousands of concurrent connections, you have to split your WebSocket servers across multiple instances behind a load balancer.If User A is connected to Server 1, and User B is connected to Server 2, how do they send a message to each other? The ArchitectureUse Redis Pub/Sub as a central message distribution fabric. When a server instance boots up, it subscribes to channels matching the users currently connected to that specific instance.[User A] -> (WebSocket) -> [WS Server 1] -> (Redis PubSub Publish) -> [ Redis Cluster ] |[User B]