Critter@home Architecture


  • Contributors have the option of hosting their own data locally, thus retaining full control over it.
  • Before data is stored, it is modified via a PPI-sanitization process to replace all personal and private information (PPI).
  • Data is always stored and transmitted in an encrypted format.
  • No human apart from the contributor will ever access the raw, PPI-sanitized, data. Instead, researchers access data via a query system which only returns aggregate statistics.
  • All contact with a contributor is at her discretion and is done via an anonymizing network where contributor identities are hidden both from researchers and the Internet at large.
  • Contributors (if they so desire) can have full, fine-grained control over their data at all times via policy settings.

Query Process


  1. A researcher submits a query via the public portal.
  2. Critter clients connect and poll for new queries via an anonymizing network.
  3. The researcher's stored query is sent to clients.
  4. Patrol processes the query if the Query Policy permits, and returns encrypted results along with information on how a contributor wants its response aggregated.
  5. Aggregated results are stored and can be retrieved.

Table Structure

Field Name Type Description
no Integer Row number unique to each row
timestmp Text Time stamp of last TCP packet assembled for HTTP content. Example - for a response, the time stamp will be the last packet in the assembled buffer.
page_id Integer
tcp_session_id Integer Identifer for all HTTP content in a TCP session
browsing_session_id Integer Identifer for all TCP sessions linked together starting from a parent page to last child linked from it, over a span of 300 seconds.
source Text Source IP concatenated with Port. For example "101.145.21.42:5335"
destination Text Source IP concatenated with Port. For example "208.11.89.1:80"
http_type Text Whether the HTTP is a Request or a Response
host Text Domain name of the form "example.com", "cdn.google.com"
url Text Relative path to the Domain name of the form "/downloads/version/1/sample.exe"
referer Text The referer of the page if any, of the form "http://www.zipcar.com/referral", "https://www.google.com"
cookie Text Cookie field for Request and Set-Cookie for Response. A Response can contain more than once cookie so each is delimited by custom character " && ". For example, "PHPSESSID=dfb9d19e2245edd083ad856ae2a5d8dc; path=/", && LyndaLoginStatus=Unknown Not Logged-In; domain=.lynda.com; expires=Thu, 12-Mar-2026 09:05:39 GMT; path=/"
content_type Text If the content of response is of the type "text/html", "image/jpeg" etc.
no_children Integer Number of a href and iframe links loaded on the page
payload Text GZIP decoded ASCII payload
hrefs Text All hrefs within a tags delimited by " "
iframes Text All iframe links loaded on the page
images Text Number of images loaded on the page with formats - ".bmp, .gif, .jpeg, .jfif, .jpg, .png, .ppm, .pgm, .pbm, .pnm, .tiff"

Query Examples

No. Query Name Query Type Query Body
1 Number of requests to Apple sum
SQL: SELECT count(*) FROM parsedhttp WHERE http_type = "Request" AND host = "www.apple.com";
2 Number of tracking cookies a user gets (upper bound) yes
SELECT COUNT(*) FROM parsedhttp WHERE http_type="Request" AND cookie!="NULL" AND host NOT LIKE '%' + referer + '%';
3 Number of requests using Chrome browser histogram
SELECT COUNT(*) FROM parsedhttp WHERE http_type = "Request" AND payload LIKE "%User-Agent:%Chrome%";
4 Number of requests using iPads histogram
SELECT COUNT(*) FROM parsedhttp WHERE http_type = "Request" AND payload LIKE "%User-Agent:%iPad%";
5 Number of requests with Windows OS sum
SELECT COUNT(*) FROM parsedhttp WHERE http_type = "Request" AND payload LIKE "%User-Agent:%Windows%";
6 Number of images loaded in the longest browsing session sum
SELECT SUM(no_children) as s FROM ParsedHTTP where browsing_session_id!=0 GROUP BY browsing_session_id ORDER BY s DESC;
7 Number of responses with iframes sum
SELECT COUNT(*) FROM parsedhttp WHERE http_type= "Response" AND iframes!="NULL";