01 February, 2012

leave your home, change your name

I discovered a spelling error in last night's mission run through City of Heroes, and was urged by a love to report it. So I did:
Hero side, from Michael Binocolo, second part of the arc, this sentence: "In all likelihood, they saw the Midnighter Squad as a giant refridgerator for Rularuu." should read "In all likelihood, they saw the Midnighter Squad as a giant refrigerator for Rularuu."
It was a simple mistake, and I took the time to send the little ticket to Support, and went about my evening.

The first missive I got back told me that they'd understood:
I am escalating your petition for review and a Senior Specialist will be in contact with you as soon as possible. You should receive a reply within 24 hours.

Take care,
GM Adonis
City of Heroes Support Team
Nifty.

By the time I got the email back from "GM Adonis", it was nearly time for sleeping, so I made my goodbyes and went to bed. This morning, I awake to find another email:
The issue you are experiencing may be due to the software not working as it was designed. We will be reviewing this ticket as a bug report through our Quality Assurance (QA) Team.

If we are able to reproduce the bug you reported, we will place it on a list of things to be fixed. We will prioritize it based on its impact on the game and players. If we can't reproduce it, we may contact you to get additional information or to provide you with information regarding the game design.

Our QA Team is unable to provide status updates on bug reports. This allows them to focus on quickly addressing bug reports and other issues in the game, so that our players can experience the smoothest game play possible.

Regards,

GM Bunni3
I have some issues with this.
  • First, City of Heroes has a "GM Bunni" in the first place? Bunni? With the "i", no less. Dear gods, the horror.
  • Second, there's no software issue. It's not a software issue. It's a spelling error.
  • Third, of course they'll be able to reproduce the bug, because it's a spelling error. It will show up.
The hell, people.

But then I remembered this happened the last time, where the same form letter was sent to me, and I reacted (in my usual gentle, caring, and circumspect fashion) with a retaliatory screed reinforcing that it was a spelling error, not a software glitch; it wasn't my hardware, my system, or the install, it was a spelling error. Rawr rant rawr. And we went back and forth for three more letters, with my end getting more shrill and their end getting more confused, until I got bumped up (or sideways, or something) a level to a new handler.

And that GM wrote back saying that the first email was the standard letter everyone gets, and thanking me for being so "passionate", but that she knew what I meant, and they'd fix it.

Yes, I do feel slightly bad for making it so personal--so "passionate", as that particular GM put it--but it's still not a software glitch. It's on their end, not mine. Lag does not misspell words.

(Unless you're in SL, where lag can erase entire paragraphs, as well as random words and individual letters. But that's also live chat, not captured quest dialogue, too.)

Fox News turned in an unflattering review of the new Muppet film, and an interviewer brought it up to the Muppets at the London press conference. Miss Piggy's notable (and wonderful) quote on the entire thing: "Yeah, if they take what I say seriously, they've got a real big problem."

Of course, what was Bill Reilly's reaction? "They better watch it." Thus, now we have show hosts on Fox News warning felt puppets to watch their actions. Just let the utter, baffling stupidity of that sink in for a moment. I don't think we've had such a national level of confusion since Dan Quayle started sending baby gifts to Murphy Brown.

In the meantime, there's a new JIRA to watch. (And remember, as always, it's WATCH, not VOTE; at least until the Lindens remove the voting option entirely, just treat it as an unnecessary distraction they won't pay attention to anyway.) This time out, it's something that could potentially effect everyone, depending on where they are.

On the Magnum servers (that would be RC Magnmum 12.01.24.248357), subscribe-o-matic delivery kiosks are being forcibly throttled on the server side. The result? No deliveries are being made from them for upwards of half an hour (or longer; one observation was sixty minutes). Which would be bad enough, but the bigger problem is it's also taking offline all vendors owned by the merchant who owns the original delivery kiosk, not just that one kiosk or vendor.

To get an idea of the impact of this issue, I'll let some merchants speak for themselves:

Chrissy Ambrose:
I was one of the reporters of this issue. It is a major problem for me, as sending inventory to the mailing list, ie notecards, induces people to come and use llGiveInventory objects, such as vendors, gift givers, landmarkers and notecard givers, all of which have stopped working by the time they arrive.

I tested the mailing list on a standard sl server version with no difficulty, but on my other sim on RC magnum 12.01.24.248357, it gives the error message and subsiquent suspension of inventory the last 3 times used.
Fred Allandale, the original reporter of this issue:
Did a few more tests and was able to trigger this condition with llGiveInventory rates as low as 5 per second. The "excessive inventory sent" message occurred after about 90 seconds. One of my customers reported getting the problem with the inventory send rate reduced to 2/second. Once this message occurs, inventory sends are blocked for 30-60 minutes.

Testing in 3 other sims NOT running RC Magnum 12.01.24.248357 at send rates up to 20/second did not trigger this condition. That was the highest rate I tested.
Kallisti Burns:
Just observed the same failure at the rate of 3 a second on the Moonsong sim after two minutes... This was after Fred flagged system owners with details of this issue and recommended this rate as a fix.

Reducing the send rate further to 1 send a second (as two a second also seems to be failing) would mean that any message would take several hours to send to the group as a whole - not ideal when some notices (relating to weekend sales events, for example - much like the one that just failed on me) are time dependent. This is hardly acceptable.
Cincia Singh, a hostess at Smooth (and, as anyone who's ever hosted events in SL knows, she needs solid access to get the word out for events at the club!):
As an SL music venue operator we depend very heavily on the timely sending of notices and gift inventory to alert group members to music performances just prior to beginning; within 10-15 minutes. Severe throttling of delivery in this manner functionally destroys a primary means of communication between venues and guests/group members.
Keep in mind, if you're seeing this behavior in your subscribe-o kiosks, or your vendors, the Magnum code was rolled out to the whole of the grid two days back.

Ms. Singh also added this:
Security issue? And the only way to fix this security issue is to break a ton of content and negatively impact communications inworld? Without warning? Group IMs and Group Notices fail (unable to retrieve group information ...) half the time, chat lag is rampant and now this throttle? X inventory offers per owner per region is useless if it takes a club hours to get notices out of an impending event. This is going to severely impact medium to small clubs in SL.
Which again (and unfortunately, in my opinion) reinforces my personal theory that all of this, just about everything negative on the grid over the past three years at least, is an unpublicized attempt to drive anyone who's using Second Life to make money off the grid.

I can go over the long string of miscommunications and misjudgements that bring us to this point, but basically, we're here now--and to prevent some sort of griefing (which most of us don't even understand at this point), this throttle of an expected behavior has been added to the server code. Blocking a vendor or a kiosk from sending inventory offers over a certain number, per person, per region means that there's going to be a lot of people who stop hearing about club events, art shows, meetings, community panels, research groups, dances...the list can and does go on. Now this, which will, at the least, impact everyone who owns a business that operates to less than one thousand members, whenever they need to send out a notice about a sale, an event, a show. (Technically, it also affects businesses that have over a thousand members, but frankly, they can generally pay hundreds of thousands in advertising, which won't have the same throttling effect.)

CarpeDiem Turbo:
I have called to complain, but we all know that will likely not do any good. I have filled a ticket for it as well. This is unreal how they can roll out an issue like this to main server KNOWING the issue it has and problems it can cause. THANKS A LOT LL.
Ivey Deschanel:
This is going to MAJORLY and for some, tragically effect 1000s of businesses in SL with subscriber services. It should be a first priority.
TheAzureShade:
i've already seen this effect a large business in SL which in turn effected me when i wasn't able to recieve my item from that business.. so not only will this effect the businesses.. but the customers as well.. most customers are not going to realize these issues are because of a throttle on give inventory.. they are going to consider this an issue with the businesses themselves.. driving revenue down.. this includes marketplace sales.. which will effect LL profits too.. please fix this FAST
CarpeDiem Turbo again:
Your right cause it's not only the subscribers that this is an issue for, a store that has a high volume sales ( like mine ) causes issues for delivery of product, ANYTHING that has a llgiveinventory will be hurt, nc givers,PRODUCT SERVERS, Magic boxes, subscribers, and so much more. My store has been effected by this shutdown cause of throttle 2 times already today and I expect more to come. Each time this happens it shuts of delivery for 30-60Min and in that time I loose the ability to have products my shoppers buy be delivered. For many merchants and shoppers this will cause a MAJOR issue. Please fix this and roll be back to previous server vrs until this is fixed as this will cost my shoppers and I grief and in the end will cost YOU LINDEN LABS money as well as me.
Many of these merchants I know, at least by virtue of shopping at their stores, and most of them I know fairly well, in terms of what they offer (and the amount of Lindens I've spent per establishment). And reading through this, taking down comments, matching names to SL Marketplace links (which I've done for everyone save Ms. Singh), I started developing a slight thread, a merest glimmer, of hope that this might just be an oversight.

Because we're not talking Mitzi9000 Resident, who was born nine days back, set up a cube in a sandbox and is retailing freebies, here. We're talking established businesses, a few of them quite large, and in at least one case, a business which owns the sim it's on (and a few surrounding ones). These are not people that the Lindens will ignore...right?

How soon we forget the lesson of Azriel Demain, banned for an impossibly spurious reason, just before rent was due on his four sims, putting not only his entire business in SL at risk, but his home on and off the grid...because Kelly Linden popped up next in the chat:
Unfortunately some mailing list and product updaters may break or need to be updated. To stop a griefing mode that has effects on the entire grid's back end infrastructure a throttle was added to llGiveInventory. This throttle matches (but is separate from) the existing throttle on llInstantMessage and exists for nearly identical reasons. That throttle is 5k per hour per owner per region; the maximum burst is 2.5k. It is impossible to hit this limit with a single script, but systems designed to spam very large amounts very rapidly may hit it and need to be adjusted. We will be monitoring the effect of this throttle to adjust it as we can if needed.
Oh. Well, then. So it was intentional.

Which means you're not going to change it.

Ever.

Wonderful.

No comments:

hide away, they say, 'cos we don't want your broken parts

Yeah, so...remember that thing I was recovering from? You know, last year ? Yeah. I did it again. So this is Em Faw Down Go Boom part ...