Scoring model

[uigrad]

Admin can correct me if I'm wrong, but I think that there is no way for us to see all the solve times on a puzzle.

Some people play the puzzles without being logged in, so they don't appear in the leaderboards, but their results are (probably) still factored into the data for that puzzle.

[Luckr]

Ah ok thanks, stoked to see it work properly!
I was just confused why I would get less points than someone else on the same puzzle while I had a quicker solve.

[admin]

Again just to reiterate: the scoring system (for points) is currently just a placeholder that is about to be swapped out with a more intuitive system. All our puzzle sites score results in comparison to previous solves. The problem is that Conspiracy Puzzles are too new to have sufficient solve data to base this sort of scoring system on, so right now the interim algorithm has to make a lot of assumptions and can occasionally throw up seemingly strange scores. This will all be fixed in the next update.

[Slugman]

Well that's backwards! I mean I get that I got low points for first place because my time vs the average time...but your third place is 72% of the average, you should have gotten much.

Ok, I give up and have no idea how the scoring works lol

[Luckr]

Ok, now I'm very confused as to why I get 306 points for 3rd place out of 5 fast completions:
image.png

[Slugman]

I know what you mean...only 165 points for first place:
image.png
but my time was 47% of the average of everyone else's

[Luckr]

Ah ok thanks for the explanation. Makes sense then why the scores are so different, because the average solution time is very highly impacted at every complete with such a small completion rate.
Feels a bit unsatisfying getting very low scores for what feels like quick solves, but I understand it's very hard to create a well working scoring system with different puzzle difficulties in the mix.

[Slugman]

More often I see people posting their scores and I just don't understand but I seem to get a lot less points than other people very often.
It also seems like it depends on the time of day, as sometimes I constantly get 300+ points for easy completions, and sometimes I keep getting 190 points for a 30 second complete at a decently hard puzzle.

I can't speak for the Admin but awhile back they did say that the scoring model was adjusted to be based on your time vs the average time of all other who have completed the same puzzle. In your example where Astrid is guilty, your time was 29% of the average of everyone else's times but UltraVenia time was 27% of the average of everyone else's times that had completed that puzzle before August 22nd...but I agree, 58 points (or 20%) less for only 2% time seems extreme.

[Luckr]

Another example. How are the differences this big?
image.png

[Luckr]

More often I see people posting their scores and I just don't understand but I seem to get a lot less points than other people very often.
It also seems like it depends on the time of day, as sometimes I constantly get 300+ points for easy completions, and sometimes I keep getting 190 points for a 30 second complete at a decently hard puzzle.
image.png

[admin]

Correct - and we will soon be updating the scoring algorithm to more accurately reflect real-world solving averages (right now it's just our system's "best guess" determining how difficult a particular puzzle is). Luckily the site has been popular enough that we're getting tons of real-world solving data really quickly - usually it takes up to a year before we can update the scoring algorithm off actual solve data.

[mellav1]

I believe it has to do with the difficulty level of the puzzle, not with whether you mark off all the innocents.

[dotnich]

Somewhat confused at the points I get from solving different puzzles. Does the number of people explicitly marked "Innocent" change the score you get, if the times are otherwise the same?