Network Trail — Boris Novichkov

The problem

You can do this by hand. Open a profile, open their friend list, open thirty of those, notice that eight went to the same school, that four of those also went to the same university, that the ones who did are the ones posting from the same city. It works, and it takes an afternoon per person, which is why nobody does it except when the stakes are high enough.

NTrail was an attempt to make the machine do the tedious half: the crawl, the joins, the counting, the ranking of what is actually unusual about a set of people — and to leave the judgement to the person reading the result.

How it works

The interface is a query bar, not a form. GET vk.user <id> friends returns a set; that set can be split into communities, intersected with another set, or asked what its members have in common. Because every result is a set, the result of one query is the input of the next, and the analysis is a sequence of small steps rather than one big configured report.

Underneath, three problems shaped the architecture:

Rate limits. VK allows about three requests a second per token, and Instagram had no usable API at all. A single client is useless at graph scale, so requests became atomic units — method, token, parameters, everything needed to run standalone — and a worker takes them in batches, spreads them over cores and machines, caches every result, and returns them together.

Dirty data. Ages that are wrong on purpose, empty fields, hidden friend lists. So attributes are inferred from the neighbourhood rather than read off the profile: if forty per cent of a person’s friends are in one university’s graph, that is a stronger signal than a blank education field.

Structure over attributes. The most useful thing about a friend graph is not any single profile — it is the shape. Communities come out of Louvain modularity over the graph, and each community gets ranked by what its members share. The clusters in the screenshot above were a school, a university course, a workplace and a home town; none of them are stated anywhere on the profile.

What it took to publish it

The project lived in six repositories with submodules pointing at each other, and in two of them a committed env_file carried a VK application secret, an access token and a database password. Publishing it meant:

rewriting every history to purge those values, and revoking the tokens;
merging six repositories into one with git filter-repo, each into its own directory, so all 470 commits survive the move;
removing the product screenshots entirely — they were captures of real profiles, with names and faces of people who never agreed to appear in someone’s portfolio. The graph above is the one artefact that shows the work without showing anybody.

What I would do differently

The domain layer and the worker were the right split, and both would survive a rewrite. The Flask backend accumulated query-planning logic that belonged in the core, and the front end talked to a hard-coded API host, which is exactly the kind of detail that stops a project from being deployable by anyone but its author. The analysis service arrived late and never fully replaced the backend it was meant to replace — the cost of building a second thing before the first one is finished.