hm... Yes, someone else mentioned this yesterday. I had just updated the database software. I didn't think it could cause this, but maybe it has.
I believe it's related to character encoding. There's a standard set of characters (letters, numbers, punctuation) that is called ASCII. It's old and works well, but lacks variety and many less common characters. The other person who reported this had used and ellipses (...) characters (rather than 3 periods) and it got messed up (saw a ? too). I think the ? shows when there's some character encoding issue.
Quotes have the same potential problem. ASCII has single and double quotes, but they are straight and lack the curved variations (true punctuation has the quotes curving differently on either side of the words they contain). So, some modern operating systems (mobile devices) use the non-ASCII characters now. Those are the ones that can end up being ?'s.
I think the database may be involved as it may not have the right character encoding to store them ... or something like that.
I'll try to look into it... maybe take some time.
A workaround is to try to mostly use ASCII characters as much as possible.
Am seeing this in other people's posts, expecting it'll change the 2 single quotes here as well. Interesting issue to deal with Bill. Surprised you're not using Unicode vs. ASCII.
I'm looking at the database stuff and not finding what might have changed.
Also, that it's even more complicated as they call the character encoding I use "latin1". Which is not just ASCII (has a lot of other special accented characters and such). So, I think it would still be fine.
There's also a layer in my own code that trying to "fix" some of the common stuff... maybe that somehow broke. But, it's hard to tell what upgrading the DB software changed.
And, there's some danger here if I try converting to another character set encoding... I could trash all the data. Hopefully, that's unlikely.
I've haven't used UTF-8 (now synonymous utf8mb3 in the database world apparently). The "latin1" encoding I use has some advantages as it uses just one byte per character. It's often faster because of that. I guess if I just switch to UTF-8, the database could become quite slow in seemingly random ways. I think since we're 99% English, latin1 is more than enough. So, sticking with that seems like a good idea still. I'm still reading about it, though.
Well, I haven't tracked down what changed or why, but my best guess is the database software is now (since updating) more strict about what it allows to be stored. So, it used to be that people would send special characters that were not in the default "latin1" character set and it would save them in the database regardless. Now, the characters that aren't in "latin1" are changed into ?.
What I can do is convert such characters to the blander variants that are in "latin1", so it will work. But, this will mean finding all the places where stuff people type gets saved in the database (e.g. forum posts, pms, offers trades, bio, ...) There are a lot of places, so that may take a while and I could still miss some. Still, it could allow for other ways to fix inputs, which could be useful in other cases as well.
I should keep looking for the cause, though. There may be some way to just do this automatically.
I always had issues with this crap on Android. Think I made a topic about it back in the day too, maybe. At some point you tightened the screws somewhere and it took care of it. Actually I think my issue may have been with apostrophes turning into & or something thinking on it again
Yes, I noticed the other day than an o with an umlaut, which showed normally before the database update, was changed initially to some gobbly gook (some random characters). I tried re-entering it, and when I saved, it changed to a question mark.
I feel like this is GameTZ showing its age. It still uses this old character set that used to be what everyone used (20 years ago). Now, not so much. But, I'm not sure I can just magically move to the new normal without the site being shredded in the process because it was all built on that old character set. Losing sleep over this... RIP GameTZ.
Some notes to my future self (and the tech-curious)...
I'm still not sure what caused it to break exactly, but there are some variables the DB uses to set character set. I think they had been latin1 by default, but for the web code I have, (after the update perhaps - can't be sure) it was utf8mb4 and that caused issues (the ? we were seeing) because I'm using latin1 in my db and it turns utf chars into ?s. So, I now set the character set before using the database from my web code to latin1 and that seems to make it work like it used to.
I should also note that all the bad ?s in entered text for the last couple days stored the ?s, so it's still like that. I don't have any great way to fix all those. So, I was just going to not fix those. From now on, it should store them correctly, though.
So, it seems fairly stable for now, but I feel more and more like I should move the whole site to UTF-8. I'll spend more time playing with it on my test server. If it seems good, I may push it out to the live site. We'll see.
