How Ulkaa Ranks The Home Feed | S Technologies Engineering

The shape of the pipeline

The home feed is a pipeline of small, separate steps. Each one is bounded in work, each one can be tested alone, and the ranking steps are pure functions with no input or output of their own. The expensive, fallible parts (the database, Redis) sit at the edges.

1.Candidates
One bounded query, one capped branch per source
2.Score
Engagement and age, affinity, interest, penalties
3.Top K
Bounded min-heap, with a share held for suggestions
4.Diversify
Author spacing and suggestion rhythm
5.Privacy
The one visibility decision, applied last
6.Session
Ids stored; signed cursor pages through them

Any Redis failure on the way drops the request back to the chronological feed.

The ranked feed is switched by an admin flag. When the flag is off, when Redis is unavailable, and for the pure connection feed, the same endpoint serves a plain chronological page. Ranking is an improvement on top of a feed that always works.

Candidates are bounded on purpose

A feed page starts from one SQL statement: a UNION ALL with one branch per source. The sources are the viewer's network (themselves, people they follow, friends, liked Pages), the groups they belong to, their workspaces, and suggestions from outside their network. Every branch carries its own LIMIT.

Where candidates come from

Illustrative values

Friends and followed. The viewer, people and Pages they follow, friends and liked Pages, each on their own timeline.

Each source is its own branch of one SQL statement with its own limit, so the bar can never be filled by one source. The proportions here are made up for the picture.

No source can starve another. A very active group cannot push every friend out of the candidate set, because it only ever gets its own branch's slots.
The ranker works on a small set. Candidates are slim rows without post bodies, so ranking in memory is cheap and its cost is predictable no matter how large the network is.
Nothing is counted. There is no total, no offset and no page number. A feed that does not know its own length never has to compute it.

Suggestions come from two places: posts matching the viewer's interests or liked by several of their friends, and the hottest posts overall (see the heat section below). The second needs no history, so a brand-new account still gets suggestions.

From candidates to the first page

Illustrative values

1.Candidatesn
A few hundred slim rows from one bounded query.
2.Scoredn
Every candidate gets one number; nothing is dropped yet.
3.Top KK
A bounded min-heap keeps the best K, a share held for suggestions.
4.Diversity pass≤ K
Author spacing and the suggestion rhythm; misfits wait for the timeline.
5.Privacy check≤ K
One visibility decision removes anything the viewer may not see.
6.First pagepage
The first slice is sent; the rest waits in the ranked session.

Bar widths are symbolic: n is the candidate count, K the size of the ranked list. Only the order of the narrowing is real.

The score is a product

Each candidate gets one number. It is a product of independent factors rather than a sum, so a weak factor pulls the whole score down and no single signal can carry a post alone:

S \;=\; B \cdot R \cdot \left(c_0 + a\right) \cdot \left(1 + b\,i\right) \cdot \prod_{j} p_j

(1)

Five factors multiplied: if any one is small, the whole score is small.

base $B$ combines engagement and age. Engagement is damped with a logarithm so the first few reactions matter much more than the thousandth, and an exponential age decay lets new posts overtake old ones. Different engagement kinds count for different amounts.
affinity $a$ is how much the viewer engages with this author, normalised, plus a flat bonus for friends. The constant $c_0$ keeps the factor above zero, so a stranger is demoted, never erased.
interest $i$ is how well the post's hashtags and category match what the viewer has engaged with lately; $b$ sets how much it can lift a score.
penalties $p_j$ multiply the score down for a post the viewer has already been shown, one they already reacted to, and one from outside their network.
retention $R$ applies to videos only and is neutral until a video has enough views; see below.

B_{\text{network}} \;=\; e^{-\lambda\,\Delta t}\,\Bigl(1 + \ln\bigl(1 + \textstyle\sum_{k} w_k\, n_k\bigr)\Bigr)

(2)

Lifetime counts of each engagement kind, weighted and log-damped, times an exponential age decay.

B_{\text{suggested}} \;=\; e^{-\lambda\,\Delta t}\,\bigl(1 + \ln(1 + h)\bigr)

(3)

For a suggestion the lifetime sum is swapped for its current heat h, so a post taking off now beats an old pile of reactions.

Anatomy of one score

Illustrative values

BaseB
some engagement, a few hours old
Retention× R
neutral: not a video
Affinity× c₀ + a
boost: a friend you engage with
Interest× 1 + b·i
small boost: a topic you sometimes read
Penalties× ∏ pⱼ
none apply

Final score, side by side

Friend's post

Suggestion

A strong suggestion still lands below an ordinary friend's post. That is intended, and it is the root of the starvation bug below.

Each row multiplies the one above it. Coloured growth is a factor above one, a red hatch is what a factor below one takes away. The multipliers are invented to show the shape; they are not Ulkaa's values.

rank.ts

export function scoreOf(c: RankInput, s: RankSignals): number {
	let score =
		baseOf(c, s) *
		retentionFactor(c) *
		(RANK.affinityBase + affinityOf(c.author, s)) *
		(1 + RANK.interestBoost * interestOf(c, s))
	if (s.seen.has(c.id)) score *= RANK.penalty.seen
	if (c.reacted) score *= RANK.penalty.reacted
	if (c.source === 'suggested') score *= RANK.penalty.suggested
	return score
}

Affinity, interests and the set of recently seen posts live in Redis as small hashes and sets that expire by themselves, each rebuilt by one bounded SQL aggregate when it is missing. Nothing about ranking is stored in PostgreSQL, and there is no view table to maintain.

Retention as a shrunk rate

For a video, the share of viewers who watch long is a signal, but a raw ratio lies on small numbers: two long views out of two is not better than nine hundred out of three thousand. So the rate is shrunk towards a prior before it is used:

\hat r \;=\; \frac{L + \alpha}{V + \alpha + \beta}, \qquad R \;=\; \operatorname{clamp}\!\left(\frac{\hat r}{\bar r},\ R_{\min},\ R_{\max}\right), \qquad \bar r = \frac{\alpha}{\alpha + \beta}

(4)

Few views keep the estimate near the prior mean; many views let the data speak. The ratio to the mean is clamped, so retention can only nudge a score.

This is the posterior mean of a Beta-Binomial model. Treat each view as a trial that is long with an unknown probability

r

, and put a

\mathrm{Beta}(\alpha, \beta)

prior on it. After

L

long views out of

V

, the posterior is

\mathrm{Beta}(\alpha + L,\ \beta + V - L)

, whose mean is exactly

\hat r

. The prior acts like

\alpha + \beta

views already seen, and below a minimum view count the factor stays at one.

Heat: forward decay in a sorted set

For suggestions, lifetime totals are the wrong signal: an old post with a large pile of reactions beats a post that is taking off right now. The base for a suggestion is instead heat, an engagement counter that fades with a half-life of a few hours.

The naive way to build it is to store a value and a timestamp per post and decay on every read, or to run a job that keeps multiplying every score down. Both are wasteful. Forward decay (Cormode et al., 2009) turns the problem around: instead of shrinking old engagement, grow new engagement.

heat.ts

// An engagement of weight w at time t, against a landmark t0.
export function forwardWeight(w: number, t: number, t0: number, halfLifeMs: number): number {
	return w * 2 ** ((t - t0) / halfLifeMs)
}

// The heat at time now of a stored score.
export function heatAt(stored: number, now: number, t0: number, halfLifeMs: number): number {
	return stored * 2 ** (-(now - t0) / halfLifeMs)
}

S \;\mathrel{+}=\; w \cdot 2^{(t - t_0)/H}

(5)

Writing: an engagement of weight w at time t is grown by how far it lies after the landmark, then added to the stored score.

h(\tau) \;=\; \frac{S}{2^{(\tau - t_0)/H}} \;=\; \sum_{e} w_e \cdot 2^{-(\tau - t_e)/H}

(6)

Reading: dividing by one shared factor gives every engagement its ordinary decay since it happened.

Every stored score is divided by the same factor when it is read. Since the factor is identical for all posts,

h_A(\tau) / h_B(\tau) = S_A / S_B

for any

\tau

: the order of the sorted set never changes as time passes, and nothing is rewritten on a read. Asking for the hottest posts is a single ZREVRANGE; decaying the few scores that come back to the present is trivial arithmetic.

Forward decay: stored scores stay, the reading moves

Illustrative values

Time now4.00H

Stored: 38.8; changes only on an engagement
Read factor: 0.06; same for every post
Heat now: 2.42; stored × factor

$h = S \cdot 2^{-(\tau - t_0)/H}$

Top: the post's heat as a reader would see it, with each engagement as a spike. Bottom: the number Redis actually stores. Move the time slider: the stored number only changes when an engagement arrives; the read-time factor alone makes the heat fade. Time is in half-lives H; weights and times are illustrative.

The landmark and the daily rebase

A stored score grows exponentially with the distance from the landmark t0, so it would eventually overflow a double. A daily job moves the landmark forward to the start of the current UTC day and multiplies every score by the matching factor. The order is unchanged and so is every heat value; only the stored numbers get smaller. The same pass drops entries that have gone cold and trims the set to the hottest few thousand, so memory stays flat.

f = 2^{-(t_0' - t_0)/H}, \qquad S' = f\,S \quad\Longrightarrow\quad \frac{S'}{2^{(\tau - t_0')/H}} = \frac{S}{2^{(\tau - t_0)/H}}

(7)

Moving the landmark from t0 to t0' scales every stored score by the same f, and the heat read at any time is unchanged.

Atomic writes

Adding an engagement has to read the landmark and increment the score in one step. If a rebase landed between the two, the increment would use the old landmark against a rescaled set. Both operations are Lua scripts, which Redis runs without interleaving:

heat.ts (increment, trimmed)

local t0 = tonumber(redis.call('GET', KEYS[2]))
if not t0 then
	redis.call('SET', KEYS[2], ARGV[5], 'NX')
	t0 = tonumber(redis.call('GET', KEYS[2]))
end
local inc = tonumber(ARGV[2]) * 2 ^ ((tonumber(ARGV[3]) - t0) / tonumber(ARGV[4]))
local score = redis.call('ZINCRBY', KEYS[1], inc, ARGV[1])

heat.ts (rebase, trimmed)

local factor = 2 ^ (-(new - old) / tonumber(ARGV[2]))
redis.call('ZUNIONSTORE', KEYS[1], 1, KEYS[1], 'WEIGHTS', factor)

Deduplication

Heat measures how many different people engage, so one person repeatedly reacting and removing the reaction must not push a post up. A reaction, save or share is counted once per post, person and kind for a few days, using a marker key set with SET ... NX EX. The first writer wins and later ones are ignored. Comments are not deduplicated, since several real comments from one person are real engagement. Nothing ever decrements heat, so deleting an engagement cannot be used to game the order either.

Top K with a bounded min-heap

A page needs the best

K

of

n

candidates, not a full sort. A min-heap of size

K

holds the best seen so far with the weakest of them at the root. A new candidate is compared to the root only: if it is better it replaces the root, otherwise it is skipped. That is

O(n \log K)

time and

O(K)

memory instead of

O(n \log n)

for a full sort, and a final sort of the

K

survivors costs only

O(K \log K)

.

rank.ts

export function topK<T>(items: Iterable<T>, k: number, compare: (a: T, b: T) => number): T[] {
	if (k <= 0) return []
	const heap = new MinHeap<T>(compare)
	for (const item of items) {
		if (heap.size < k) heap.push(item)
		else if (compare(item, heap.peek() as T) > 0) heap.replaceTop(item)
	}
	return heap.toArray().sort((a, b) => compare(b, a))
}

Top K with a bounded min-heap

Illustrative values

42
17
63
8
55
71
23
90
12
48
66
35

12/12Candidate 12 scores 35. It does not beat the root, 55, so it is skipped after one comparison.

Final sort of the survivors, best first: 90 > 71 > 66 > 63 > 55

K = 5 and the scores are illustrative. The root (ringed) is always the weakest of the best K so far, so each newcomer needs one comparison to be skipped and O(log K) work to be kept.

The comparison is a total order: higher score first, then newer, then the larger id. Two posts never tie, so the same input always gives the same feed, which makes the ranker straightforward to test.

Diversity, and the starvation bug

Score order alone makes a poor feed. One prolific author would fill a screen. So a greedy pass walks the ordered list and, for each slot, takes the best remaining post that fits the rules: the same author may not appear twice in a row, only a limited number of times in any short window, and suggestions are spaced out. A post that cannot be placed leaves the ranked list and still shows up later in the chronological continuation.

Written as constraints on the output sequence

x_1, x_2, \dots

, with

\operatorname{au}(x)

the author,

\sigma(x) = 1

for a suggestion, windows

W_a, W_s

and caps

m_a, m_s

, slot

s

may take

x_s

only if:

\begin{aligned} &\operatorname{au}(x_s) \neq \operatorname{au}(x_{s-1}) \\ &\bigl|\{\, j \in [s - W_a + 1,\ s] : \operatorname{au}(x_j) = \operatorname{au}(x_s) \,\}\bigr| \le m_a \\ &\textstyle\sum_{j = s - W_s + 1}^{s} \sigma(x_j) \le m_s \end{aligned}

(8)

Never the same author twice in a row, a cap per author in a sliding window, and a cap on suggestions in a shorter window.

The bug: suggestions never reached the top

Suggestions are deliberately penalised, because a stranger's post should lose to a friend's post of similar quality. Combined with the affinity factor (friends get a bonus, strangers get none), that made a suggestion score systematically lower than a network post. The top K was cut by score first, so with a healthy network the cut-off sat above every suggestion. The spacing rule in the diversity pass had nothing to space, because no suggestion survived to be placed.

The fix: a reserved share and a slot rhythm

Two changes work together. First, a share

\rho

of the top

K

is held for suggestions: the best

\lceil \rho K \rceil

suggestions are selected among themselves, and only the remaining

K - \lceil \rho K \rceil

places are filled by overall score. Second, the diversity pass prefers a suggestion on every slot with

s \equiv 0 \pmod{r}

, falling back to the best fitting post when none is available, so the reserved suggestions are spread through the page and not bunched at the end.

Twelve slots, before and after the fix

Illustrative values

Network only

F
G
F
F
W
F
G
F
F
F
G
F

A healthy network fills every slot by score. Fine, until suggestions should appear too.

Before: suggestions starved

F
G
F
F
W
F
G
F
F
F
G
F

Below the top K cut, never placed:SST

After: reserved share and a slot rhythm

F
G
S
F
W
T
F
G
S
F
F
T

The best suggestions are held before the cut, then placed on every r-th slot; the network still holds most of the page.

Friends and followedGroupsWorkspacesInterest suggestionsTrending

Each square is a slot on the first page, best first. The rhythm is drawn as every 3rd slot (ringed) for the picture only; the real share and rhythm are tuning knobs and are not published.

rank.ts

// Suggestions score below friends' posts by design, so hold a share of the top K for the best of them.
const held = topK(suggested, Math.ceil(k * RANK.suggestionShare), compareRanked)
const heldIds = new Set(held.map(c => c.id))
const rest = topK(scored.filter(c => !heldIds.has(c.id)), k - held.length, compareRanked)
return diversify([...held, ...rest].sort((a, b) => compareRanked(b, a)))

// In diversify(): on a suggestion slot, prefer the best suggestion that fits.
if (out.length % RANK.suggestionEvery === RANK.suggestionEvery - 1) {
	index = remaining.findIndex(c => c.source === 'suggested' && fits(c, out))
}
if (index < 0) index = remaining.findIndex(c => fits(c, out))

The penalty stays, so within the reserved share the strongest suggestions win, and a viewer with a rich network still sees mostly friends. The share and the rhythm are tuning knobs that we adjust from experience, which is why this guide names them but does not publish them.

One privacy decision, applied last

Ranking decides what is interesting. It never decides what is allowed. Every post that is about to be shown, whether it comes from a fresh ranking, a stored session or the chronological tail, goes through the same function, decide, which knows about audiences, blocks, mutes, group secrecy and sandbox accounts. The SQL pre-filter inside the candidate query is only an optimisation and is built to be never stricter than decide, so it can only remove rows the decision would remove anyway.

A stale list is safe. If someone is blocked after a ranked list was stored, the next page re-checks every id and the blocked author's posts simply disappear. No cache invalidation is needed for correctness.
Ranking code cannot leak. A bug in scoring may produce a worse feed. It cannot produce a visible private post, because scoring is not where visibility is decided.
A failed lookup throws. If the relations needed for a decision cannot be loaded, the request fails; content is never shown on a guess.

Signed cursors and the ranked session

Ranking is not stable between requests: heat moves, new posts arrive. Paging by re-ranking on every page would repeat and skip posts. So the first ranked page stores its ordered ids as a session in Redis, and later pages just read the next slice of that list. The list is short-lived and expires on its own.

When the session runs out, the feed continues chronologically from the top and excludes every id the session held. A reader who scrolls far enough gets from the ranked part into a normal timeline without a repeat and without a seam.

The client only ever holds an opaque cursor. It has two shapes, ranked and chronological:

cursor.ts

export type ChronoCursor = { m: 'c'; t?: string; id?: string; s?: string }
export type RankedCursor = { m: 'r'; s: string; i: number }

function sign(body: string, binding: FeedCursorBinding): string {
	const bound = JSON.stringify([binding.feed, binding.viewer, binding.filter ?? ''])
	return crypto.createHmac('sha256', cursorKey()).update(`${bound}.${body}`).digest('base64url')
}

Signed. A cursor carries an HMAC, so a client cannot edit it to jump into someone else's session or to read an arbitrary position.
Bound. The signature covers the feed, the viewer and the media filter. A cursor from another feed or another person fails verification.
Keyset, not offset. The chronological shape stores the last (created_at, id) pair and asks for rows strictly below it. The timestamp keeps the database's own precision so rows sharing a millisecond are not skipped.
Expiring. A cursor stops working after a while and the client is told to start again.

Failure behaviour

Redis makes the feed better; it must never make it fail. Every Redis call in the feed path has a short deadline and swallows its own errors. A heat write that fails is logged at debug level and the request goes on. A signal read that fails turns the ranked path off for that request.

heat.ts

} catch (error) {
	logger.debug('Could not record post heat', { error, kind })
	return false
}

How the feed degrades

Ranked feedHealthy
Flag on, Redis answers: candidates, signals and heat are all readable.
heat read fails
Ranked, without heatDegraded
Suggestions fall back to their lifetime-engagement base. The page is still ranked.
Redis down or slow, a signal is missing, or the flag is off
Chronological feedFallback
Same endpoint, same response shape, newest first. Ranking was an improvement; this always works.
privacy relations cannot be loaded (any path)
Request failsRefused
The relations for a visibility decision could not be loaded. Content is never shown on a guess.

Every Redis call has a short deadline and swallows its own error, so a Redis failure only moves the request down to a working rung. Privacy is the exception: it fails closed.

What failed	What the reader sees
Redis is down or slow	A chronological feed, same endpoint and same response shape
Heat cannot be read	Suggestions keep their lifetime-engagement base; the page is still ranked
The stored session expired	The feed restarts chronologically; the client drops repeats by id
The ranking flag is off	Chronological, the same as a Redis outage
The cursor is invalid or expired	A clear 400, and the client starts again without a cursor

The one thing that does not degrade quietly is privacy: if the relations for a visibility decision cannot be loaded, the request fails rather than guessing.

What we have not solved yet

Negative feedback. There is a penalty for posts a reader has already seen or reacted to, but no first-class "show me less of this" signal feeding back into affinity and interest.
Impression-based ranking. Engagement is measured against all posts, not against how often a post was actually shown. A post that was shown rarely and engaged with often is under-rewarded, and a post shown to everyone is over-rewarded.
Tuning from real data. The weights, the share held for suggestions and the half-life began as reasoned starting points. They need to be fitted against real behaviour, with offline replay and controlled experiments, before we can say they are right.

Negative feedback, language and topics are now covered by a second layer on top of this ranker; see How Ulkaa Personalises The Feed Cheaply.