Public methodology · Version 1

How grades work.

How subscription scores work

Every review requires email verification. Plan ownership, amount paid, and subscription status come from the reviewer. We check favorable and unfavorable reviews under the same content policy.

• Satisfaction is the average of eligible overall ratings from experiences in the past 90 days.
• Paid plan value has its own average. Reviewers can skip price and renewal questions if they did not personally pay or prefer not to answer. Unanswered value and renewal responses are left out of their respective averages and percentages. Optional feature feedback is shown separately. Complaint and outage reports are counted separately.
• Each account contributes one approved review per plan. Edits are moderated and replace the previously approved contribution. We keep revision history for moderation.
• We label scores with fewer than 20 contributors “Early feedback.” Reviewers choose whether to contribute, so the results describe these reviewers.
• Renewal intentions and willingness to subscribe again are reported separately, with the number of respondents in each group.

Older published experiences remain readable after they leave the scoring window. Earlier bot feedback uses the legacy method below and stays separate from subscription scores.

Weekly subscription votes

An email-verified account can submit one satisfaction score per plan per Eastern voting week, from Monday at midnight through Sunday night. If you update a vote during that week, only your last score counts. Test votes are excluded.

Weekly results show the number of votes and, when eligible for publication, their average on a 1–5 scale. The aggregate results publish automatically at 9 a.m. Eastern Monday. Quarterly totals use all individual votes from published weeks that began in the quarter, so a week with more votes contributes proportionally more. An account can vote in more than one week, so quarterly counts are votes, not unique people. Written reviews have their own moderation and scoring rules.

Votes are voluntary and self-selected, not a representative sample of all users. Results describe participating voters, not all customers or product quality overall.

Scores are withheld for plan-weeks with fewer than two votes. A quarterly score is also withheld if any contributing week has a withheld score. Vote counts remain visible, and all votes still count toward the quarterly turnout. Suppression does not guarantee anonymity: known votes or other information may still reveal a score. It cannot recall previously downloaded or published data.

Legacy bot report card

Only verified user ratings and eligible complaints submitted to RateTheBots feed the scores. We do not scrape reviews or social posts.

The window and the minimum

Reporting weeks run from Monday at midnight through the end of Sunday, Eastern time. Weekly rating counts include each distinct user/bot rating submitted or edited during that week once, with collected, eligible and held counts shown separately. RTB Scores use eligible ratings in the rolling 28 Eastern calendar days ending with the reporting week. Nothing before the launch on November 9, 2026 at midnight Eastern counts. Test submissions never count. Only owner-approved reports publish at 9 a.m. Eastern on Monday. A bot needs at least 15 weighted ratings; otherwise it shows “Not enough ratings yet.”

Converting answers

Each 1–5 answer becomes (answer − 1) / 4 × 100. The mistakes question already has 5 as the good answer; it is not reversed again. N/A answers are skipped.

Response weights

• Start at 1.0; approved verified subscribers start at 3.0.
• Usage tenure under a week multiplies weight by 0.5.
• Risk scores 40–69 multiply weight by 0.5.
• Pending-email, held and excluded ratings have weight 0. Risk 70+ stays held until approved.

The five components

Satisfaction · 30%

70% overall satisfaction plus 30% recommendation answer rescaled from 0–10 to 0–100.

Reliability · 20%

Mean task success, unattended work and mistakes answers; subtract 10 points × the share reporting an outage.

Privacy and safety · 20%

Mean privacy and approval answers; subtract 15 points × the share reporting an unrequested action.

Value · 15%

Mean value and usage-limit answers; subtract 10 points × the share reporting unexpected charges or a limit/price change. Self-hosting uses setup instead of value.

Complaint resolution · 15%

All eligible, verified non-safety complaints older than 14 days contribute. Only resolved/unresolved answers enter the denominator. (resolved + 5) / (answered + 10) × 100.

Available answers are averaged within each rater, then raters are weighted. A wholly missing component uses 50. Clamp every component to 0–100.

raw = 0.30 × Satisfaction + 0.20 × Reliability
    + 0.20 × PrivacySafety + 0.15 × Value
    + 0.15 × Resolution

Small-sample smoothing

The prior is the mean raw score of all bots with enough data, or 60 if none qualifies. With k = 10:

score = (weighted_n × raw + 10 × prior)
      / (weighted_n + 10)

Letter grades

Integrity and uncertainty

Ratings require email confirmation and abuse checks. Unusual activity can hold new accounts for review. Receipts require owner approval. User reports describe experiences, not independently established findings. A grade is not a safety guarantee.

Method changes

• 2026-11-06: Initial method. Available answers are averaged within each rater before weighting; wholly missing components use 50; resolution includes all eligible non-safety complaints older than 14 days.

Contact: press@ratethebots.com