Every year, around the third week of November, a quiet crisis begins inside e-commerce companies that thought this would finally be the year things held together. Traffic climbs. Slowly, then all at once, the ingestion queues stretch. Recommendation engines are already surfacing products that were sold out three hours ago. By Friday morning, the engineering team isn’t watching conversion rates climb. They’re on a call nobody wanted to schedule, working through logs from a system that technically did everything right and still somehow failed.
The breakdown rarely traces back to a single point. What collapses is the layer responsible for collecting and making sense of data at scale. Companies that invest in data lake consulting often discover that November’s problems were visible months earlier, buried in metrics nobody was watching. The same holds for consulting work around lake architecture more broadly: the distance between a lake built for steady traffic and one designed to handle a 10x demand event isn’t a matter of configuration tweaks. It’s a structural decision, made or avoided long before the holiday ads start running.
The Architecture That Looks Fine Until It Doesn’t
Adobe Analytics tracked $10.8 billion in US online spending on Black Friday 2024 alone, a 10.2% increase over the prior year. Every dollar of that spend generated data events: clicks, cart additions, session timeouts, failed payment attempts, and refund requests.
Traditional data warehouses weren’t designed for that kind of ingest. Neither were lakes assembled without partitioning strategies or autoscaling ingestion pipelines. Under Black Friday load, latency climbs from milliseconds to seconds. Recommendation calls that depend on fresh session data start returning stale results. Inventory signals lag, and the marketing team, watching dashboards that stopped reflecting reality thirty minutes ago, makes decisions on numbers that are already wrong. The checkout page may still be loading. The architecture beneath it has already failed.
What makes this so frustrating is the predictability. Black Friday falls on the same Friday every year. By early October, any analytics team can model the traffic curve using data from the last two Novembers. The scale grows, reliably, by a percentage anyone can estimate using last year’s numbers.
What Good Data Lake Design Actually Does When Traffic Spikes
A lake built with Black Friday in mind looks different from one designed around typical traffic assumptions. Raw event streams land in a dedicated ingest zone. Further downstream, processed behavioral data and curated reporting tables each have different access patterns and update frequencies. When traffic spikes, the ingest layer absorbs it without collapsing the downstream queries that power inventory dashboards and recommendation engines at the same moment.
Partitioning matters enormously here. A lake that partitions by date at the table level will slow considerably when thousands of simultaneous sessions arrive. Organizing around actual query patterns, whether by event type or by the product category structure the business already uses, distributes the load and keeps read latency from compounding across layers. Autoscaling compute helps. But computing alone doesn’t fix poor data organization. Research tracking cloud failures across industries found that retail companies lose an estimated $120 million annually in aggregate from peak-season downtime, and that figure doesn’t account for the slower, less visible degradation that happens when data infrastructure bends under load without fully breaking.
N-iX, a technology services company with deep experience in data engineering and platform architecture, has worked through these problems with e-commerce clients facing exactly this kind of seasonal pressure. What tends to surface early in that work is uncomfortable: most Black Friday failures trace back to design choices made in Q1 or Q2, by teams that never imagined they’d eventually be processing the data volumes they’d see in November.
Assessing whether a lake is ready for peak traffic means examining a handful of things:
- Whether the ingestion pipeline can scale independently from the query layer
- Whether raw, processed, and consumption-ready data are stored and governed in separate zones
- Whether partition strategies reflect actual query patterns rather than theoretical ideals
- Whether monitoring detects latency increases before end users experience them
- Whether data contracts exist between upstream producers and downstream consumers
These aren’t exotic requirements. They tend to surface in the first week of any serious lake architecture advisory engagement, not because the questions are hard, but because nobody thought to ask them when the lake was first designed.
When to Bring in Outside Help
There is a particular moment in a company’s data journey when the lake that once felt like an asset starts behaving like a liability. Queries that returned in under a second now take twelve. By the time dashboards refresh, the information they display is already stale. The team that built the original structure has turned over, and the engineers who inherited it are maintaining documentation that describes an architecture from three years and several data source integrations ago. Nobody is sure what’s in the bronze zone anymore, or why it was named that.
Forrester’s analysis of enterprise data strategy found that organizations need to shift from treating data infrastructure as storage to building systems capable of dynamic, real-time engagement, which requires rearchitecting legacy setups rather than extending them. That gap, between what a lake was designed to handle and what the business now demands, tends to widen quietly. Then it announces itself during the first week of December.
Data lake consulting engagements often begin as diagnostic work. An outside team maps the current architecture against actual data flows, identifies where the structure diverges from access patterns, and recommends targeted changes rather than a full rebuild. What that process tends to surface is both obvious and easy to miss from inside the organization: the ingestion pipeline grew to meet demand incrementally, without anyone redesigning the partitioning model that was set up in year one. Repartitioning solves some problems. Separating ingestion from transformation compute solves others. Occasionally, the right answer is a thin serving layer in front of the lake, so that high-frequency queries stop hitting raw storage altogether.
Conclusion
Companies that treat Black Friday performance as a postmortem problem, rather than a planning problem, tend to find themselves in the same conversation twelve months later, with slightly different failure modes. Engaging lake architecture advisory services before the seasonal pressure arrives is considerably less expensive than the engineering hours spent during it. A data lake that holds under 10x traffic isn’t built in a week. It’s built in the months when nobody is watching the dashboards yet.
Ask Amy Glazerela how they got into market analysis and reports and you'll probably get a longer answer than you expected. The short version: Amy started doing it, got genuinely hooked, and at some point realized they had accumulated enough hard-won knowledge that it would be a waste not to share it. So they started writing.
What makes Amy worth reading is that they skips the obvious stuff. Nobody needs another surface-level take on Market Analysis and Reports, Investment Strategies and Trends, Wealth Management Strategies. What readers actually want is the nuance — the part that only becomes clear after you've made a few mistakes and figured out why. That's the territory Amy operates in. The writing is direct, occasionally blunt, and always built around what's actually true rather than what sounds good in an article. They has little patience for filler, which means they's pieces tend to be denser with real information than the average post on the same subject.
Amy doesn't write to impress anyone. They writes because they has things to say that they genuinely thinks people should hear. That motivation — basic as it sounds — produces something noticeably different from content written for clicks or word count. Readers pick up on it. The comments on Amy's work tend to reflect that.
