headscratching point values

[Naboka]

As the screen shot shows, "phosphorescent" is a 14 letter word worth only 13 points.

13!

Because it's common? !

When was the last time you used phosphorescent in a sentence? Or read it anywhere? Or heard anyone use it over lunch?

At least two very good players had done the board previously and they didn't see it. The board had been played over 30 times. So, how is so common?

Because the board stats indicated the longest word found thus far was 8 letters, when I saw the pho (6 points) and phos (6 points) with another phor attached, I was thinking phosphor-something! Good points! Has to be. And far more than 8 letters!

Had I known phosphorescent was garbage points I would have ignored it like so many common long words worth less than a point a letter.

I want a refund.

I think I'm behaving like a spoiled brat.

Screen Shot 2022-04-09 at 9.43.05 AM.png

[dannyb]

Well, in teaching biology, I would mention phosphorescent algae and squid that exhibit bioluminescence (I used the word phosphorescent in teaching chemistry and physics too. In my biology class I would mention the beautiful image from Donovan's song Sand and Foam, "The simple act of an oar's stroke put diamonds in the sea . . ."

[Naboka]

Well, in teaching biology, I would mention phosphorescent algae and squid that exhibit bioluminescence (I used the word phosphorescent in teaching chemistry and physics too. In my biology class I would mention the beautiful image from Donovan's song Sand and Foam, "The simple act of an oar's stroke put diamonds in the sea . . ."

But...

do you think the word is common? Something from every day speech?

Maybe "wide."

Or between "wide" and "rare?"

How are these words classified? By frequency of use? By general knowledge of the word?

And shouldn't words be given bonus points because of a "cool" factor? Or fascinating factor?

Just curious what you think.

[dannyb]

Probably someone, somewhere, created and algorithm and scanned written texts and such into the program and there is a data base of word usage. Let me do a quick check . . . Princeton U has WordNet database. Here is a site that is useful https://www.wordfrequency.info/ it will show you some cool info on how frequency is derived.

[lalatan]

Original post flagged as spam after editing , to .
Naboka, once upon a time, Stephen said admin bought a word usage database (a 2012 version I think) and that's the basis for the word categories. He said he wanted to update it but apparently hasn't done so. It seems he's reclusive once again. (Maybe he got too much heat when he was in the public eye.) He said last year he was going to introduce his own scoring system last month. It was going to be based on stats of the percentage of time a particular word was found vs. the number of times the board was played. I'm sure the pt values of long words will drop if he implements it since there are so many speed players now regularly finding long words.
I've also used phosphorescent in past conversations.

dannyb, that was some cool info at the Princeton link you shared. Thanks.

I hesitate to say this but finally will. Not targeting anyone in particular but it seems to me we are a pretty demanding group of players (I include myself in the past). The vast majority of us have never paid a dime for the use of this website. If it were my business I'd probably pay more attention to paying customers' complaints. I know it's irritating when there are missing words and scoring anomalies (like DEINSTITUTIONALIZATIONS being worth 38 or 36 pts and INSTITUTIONALIZATIONS being worth 43 pts). Eventually, I became worn out by newbies griping about the same missing words over and over in the Words not accepted thread so I resolved not to do likewise. For all our griping very little ever changed regarding valid words that weren't accepted. I came to accept that it's not going to change and to just enjoy playing the game as it is, "warts and all." I'm not advocating there be no griping, just my decision that helped me be more at peace.

[Naboka]

Probably someone, somewhere, created and algorithm and scanned written texts and such into the program and there is a data base of word usage. Let me do a quick check . . . Princeton U has WordNet database. Here is a site that is useful https://www.wordfrequency.info/ it will show you some cool info on how frequency is derived.

That's pretty cool. Seems to want me to pay to subscribe. But still interesting that someone has compiled it.

A question: being a science teacher, does it bug you how many scientific terms aren't accepted here--especially when seemingly random ones are?

Like I just played a board with amimic and amimia possible...but...bzzzzzzz...no dice. Perfectly valid words. They both basically mean to lose the ability to express oneself with gestures. Maybe it's just my fingers are having a lot of trouble today.

Sigh.

[lalatan]

Naboka, I wanted to add that I completely understand your frustration and disappointment to have new, valid words rejected. I went through that many times myself. For whatever reason, Stephen doesn't have the will or possibly the resources to change the situation.

[DrPlacebo]

Original post flagged as spam after editing , to .
Naboka, once upon a time, Stephen said admin bought a word usage database (a 2012 version I think) and that's the basis for the word categories. He said he wanted to update it but apparently hasn't done so. It seems he's reclusive once again. (Maybe he got too much heat when he was in the public eye.) He said last year he was going to introduce his own scoring system last month. It was going to be based on stats of the percentage of time a particular word was found vs. the number of times the board was played. I'm sure the pt values of long words will drop if he implements it since there are so many speed players now regularly finding long words.
I've also used phosphorescent in past conversations.

And yes, just to add to this, the word usage database for this game is based on written texts, not colloquial English.

[2cute]

Had I known phosphorescent was garbage points I would have ignored it like so many common long words worth less than a point a letter.

I want a refund.
I think I'm behaving like a spoiled brat.

LOL Naboka - I had a premium membership for about 6 months - I must admit, I 'missed' the ads, they're not as annoying on this site as they are on others.

I hesitate to say this but finally will. Not targeting anyone in particular but it seems to me we are a pretty demanding group of players (I include myself in the past). The vast majority of us have never paid a dime for the use of this website. If it were my business I'd probably pay more attention to paying customers' complaints. I know it's irritating when there are missing words and scoring anomalies (like DEINSTITUTIONALIZATIONS being worth 38 or 36 pts and INSTITUTIONALIZATIONS being worth 43 pts). Eventually, I became worn out by newbies griping about the same missing words over and over in the Words not accepted thread so I resolved not to do likewise. For all our griping very little ever changed regarding valid words that weren't accepted. I came to accept that it's not going to change and to just enjoy playing the game as it is, "warts and all." I'm not advocating there be no griping, just my decision that helped me be more at peace.

Yep, we can be very demanding. Maybe Stephen took a vacation, rather than be incognito. Do you think he manages all the games or just WordTwist?

[bwt1213]

A really long time ago, WordTwist was a different game. "Q" appeared only once. Same for Z. In fact, all the letters appeared the same way as they would have for a Boggle game. The word list was pretty much SOWPODS, with just a few extensions. Stephen asked for suggestions to improve the game. I was one of a number of people who said that it would be a good idea to stop pretending that letter cubes were being shaken up and dropped into a 4x4 or 5x5 grid and to just let random letters drop, picked according to their frequency in the English language. Let six "X" appear or five "Q" or "Z". Don't worry about being so damned faithful to Boggle. And so it was, and the new game appeared. That was a really big change. And new words were added, too, because SOWPODS is stupid beyond belief and included so few actual English words. So Stephen has done some pretty drastic changes and updates. And that's why I am encouraging him to create a 6x6 version and to add even more words -- because he HAS listened, and has done a whole lot of work in the past. I feel guilty about asking him to do even more. But he has done a whole hell of a lot of work already.

[dannyb]

That's pretty cool. Seems to want me to pay to subscribe. But still interesting that someone has compiled it.

A question: being a science teacher, does it bug you how many scientific terms aren't accepted here--especially when seemingly random ones are?

Like I just played a board with amimic and amimia possible...but...bzzzzzzz...no dice. Perfectly valid words. They both basically mean to lose the ability to express oneself with gestures. Maybe it's just my fingers are having a lot of trouble today.

Sigh.

I am more bugged by non-science and non-medical terms not being accepted. Retinned (cost me a pretty penny, to get my saute pan retinned, and nage a flavored poaching liquid that can also be served with the poached item. Of course ela the highest note on the Guido musical scale (the game accepts ut [as the plural uts] which is the lowest note on the same musical scale).

Whadda ya gonna do? This is the game that we have chosen.

[admin]

Hi everyone -

The dictionary used by Wordtwist was substantially upgraded in 2020-2021, though I agree there are still improvements that can be made. Because of the way the site runs, dictionary improvements cannot be made incrementally but must be done in large batches, as it requires a site-wide reanalysis of every single game board. I do have plans underway to add another major dictionary upgrade later this year, however, which should do a better job catching medical/specialist terms. Please understand though that no dictionary will ever be perfect! ("Retinned" is a great example of this problem - I can't find a single major dictionary that specifically lists this term.)

I've also spent a lot of time thinking about word rarity/frequency scoring.

My initial plan was to switch over to "find frequency" scoring, i.e. we would score a rarity value for each word found based on how frequently it was successfully submitted by previous players on the exact board being played. To that end we've been collecting mountains of data over the last several years to track in-depth statistics on each and every word played, and at this point we have enough information to actually launch something along these lines. But this method, too, presents some interesting issues (and some fascinating insights on potential "blind spots" in Wordtwist):

Take this board as an example:

SNTAN
OIGSR
TERIT
SCRFE
RELER

So far it's been played 221 times, and the most frequently found words are:

SING --> 134
TIRE --> 125
STING --> 122
TIRES --> 121
TING --> 119
SINGS --> 119

Makes sense so far. But then here's a sampling of some of the "rarest" words found, each discovered only by one player after 221 solves:

AGIOS --> 1
REREGISTRATIONS --> 1
SCLERE --> 1
TIERCEL --> 1
REGION --> 1
GIRL --> 1
NASTIEST --> 1
SONGSTER --> 1
CRESTON --> 1
NIECE --> 1

REREGISTRATIONS or CRESTON both make sense, but GIRL??? And NIECE, which isn't even all that well-hidden (it's in a perfect vertical line in the second column)? If we were to score rarity values based on this new method, then GIRL and NIECE would both show as "ultra rare", which I'm sure would lead to confusion and several complaints about our methodology, just because they don't "feel" like rare words, or even appear to be in hard-to-find layouts on this particular board.

Sidenote: Purely on the behavioral note side of things, it is interesting (to me at least) that both of these extremely common words are missed by over 99% of players on this board, and they both appear either entirely or mostly vertical in the center portion of the board. I am tempted to run some larger statistical models to see if there's an underlying pattern of "blind spots" like this, where extremely common words are frequently missed strictly due to their position/layout on the board.

So... should we use this "found frequency" data to improve the rarity scores, or not? I'm still on the fence. My gut feeling at this point is that we might end up using three data points in some combination/weighting to define a term's scarcity: (1) the word's English language rarity, i.e. the current scores being used right now, (2) the word's "found frequency" for the board currently being played, and (3) the word's global "found frequency", i.e. of all the boards where this word can be played, how many times was it successfully found.

Anyway, that's likely more information than most would care to read, but that's some insight on what I'm working on at the moment when it comes to this game. I apologize that I don't chime in on the forums as frequently as I should - apart from Wordtwist I'm constantly adding to and maintaining all of our other Puzzle Baron games/puzzles, and I tend to get tunnel vision on one game at a time and focus my time/energy there for a bit, often at the expense of all the others. I'll try to do better with this in the future.

Stephen

[dannyb]

I think that using the "found frequency" would change the nature of the game. How often will the "found frequency" change the category a word is in? Yearly, monthly, weekly, I think it will really freak out the people that play for points for word or average score per game. The idea that oont will become a common word is difficult to wrap my head around. Certain letters near each other trigger certain finds. All of the "oo" words will become common oom, oon, roon, oot oose etc. The game will become a "hidden" word search game.

[Naboka]

Stephen,

Really appreciate the feedback. That's a lot of work. Hadn't realized the breadth of complexity for your decision making.

Here's my 2 cents, but it's personally skewed.

"Played" frequency would be different from "found" frequency. Due to low point value, I wouldn't play "girl" except as a last resort. Neither would any of the long word players. Not to mention those who play for averages and dump lots of boards and who might well have played it. So, a word may have been found far more than it's been played.

Games have a reward factor. The question here is rewarding words for rarity. Do you reward a word's rarity because knowing it requires a lot of study and hard work, or do you reward a word merely because people have trouble finding it?

Or is a word's rarity merely the accidental product of being part of a particular group? Like dannyb mentioned oont. Who would know oont except someone either living around camels or playing WortTwist?

Players who have worked hard at this game have accumulated stockpiles of high-point-producing words. Rare and ultra rare words. Which are played with great frequency here--BY SEASONED PLAYERS! Negating that advantage because "all the good players get those words" just seems inappropriate.

Found frequency simply seems like a statistical anomaly in determining rarity. Gold isn't all that rare in a gold mine. A group of chemists wouldn't blink at "stannate" or "tannate." But, for most of us...?

The statistical rarity of a word would be somewhat dependent on the sampled group.

"Blind spots" were one of the things I recognized analyzing why so many obvious words were missed. Since I only play 4x4 that problem must be greatly compounded on 5x5. My solution was to memorize scores of anagrams and use those mentally rather than focus exclusively on the board.

I hope that whatever solution you find doesn't result in unrewarding those who have worked hard at this game.

[Naboka]

And

I hope there won't be a serious devaluation of word values. Here's a random page from one of the 3 240 pages notebooks that took a lot of work and study. (getting under 2mb limited
the image and number of words shown.)


bScreen Shot 2022-04-10 at 10.31.02 AM.png

In an instant,
"stultify"
would apply.