System Updates
We've just made a major update to our blogging software, better be sure it works...
We've just made a major update to our blogging software, better be sure it works...


(Graphs are correct at time of article publication)
Post your secrets anonymously, and have them appear online. First listed in April 2006, PostSecret quickly gained value, peaking at over B$100,000,000 before being hit by the Potent Wind decline that pushed the valuation down to its current level of B$8.3m, however the blog continues to grow in strength each month, soon to pass 10,000 incoming links. The blog has a strong position in the Top 100 and the owner Frank Warren still continues to post fresh secrets every Sunday. Begun as a community mail art project, the site quickly gained a huge following, now with over 60,000 people registered on the chat forum. Frank himself regularly comments and appears to speak at venues across the US.
In other news, yesterday the Blogshares server went down (again), suffering it's first hard disk failure since 2003. We're temporarily on lowered strength until the disk(s) can be replaced and the RAID array rebuilt, hopefully (!) in the next few days. Also, it seems that for a while users have been experiencing problems using PayPal to purchase Premium Membership. We'll be looking into this, but if you have any information that could be of use, drop us an email.
I've spent the last two weeks in California, visiting with Jay and checking out the area - it's fantastic, I hope to move here very soon. During the visit, I had the opportunity to visit the hosting facility for Blogshares, and snap a photo of the old database box, shown here. In case you didn't know, it's a Quad PIII 550Mhz with 4Gb RAM and 10K SCSI disks with RAID-5. It's served us well, handling up to 900 queries per second when the spiders were running flat out alongside the website, but it's slowly failing. Not to worry, due to your contributions, soon this old box will go on easy street.
No, we haven't purchased the new one yet - this is taking some careful planning and execution, going for a balance of future security while not compromising on components. We're gonna shell out for a 3 year 4-hour response warranty out of our own pocket, on top of the community donations, to make sure that in the unlikely event of a hardware problem, we can get it sorted quickly. The hope is we'll have a box on order within the next two weeks, we'll keep you informed of any news.
Cheers!
Addendum: While doing some Java today, I'm reminded of a site that has helped me immensely in the maintenance and growth of the spiders, as well as our other projects, yet I've never given it props. It's Roedy Green's Java & Internet Glossary, a useful resource. Thanks to him for the help it's brought me.
We've had our own share of ups and downs lately with the Blogshares servers, the AboveNet NOC suffered power outages of some sort two mornings in a row, knocking us out and creating a few hours of table checking and fixing in both cases. Finally, we're back to normal.
Or so we thought. It turns out the PubSub Feedmesh has been offline for the past 24 hours, and we're worried it may not come back, after warnings a few months ago that they were about to close. This was our primary source of pings for new and recently modified blogs, and it's disappearance (again) meant it was time to try something else.
Whats our other option? Blo.gs. We've been approved to access their Cloud Interface for a few months, so we gave it a try.... after a few minutes, we were forced to turn it off again while we did some hard thinking.
Why? Here's an example of the quality of pings we were receiving - this is in raw ping format. Looks like about 90% spam, up from the more usual 30 - 50% spam we'd see from PubSub, and quite a bit of it was getting past our fairly decent filters. This is, as far as we're concerned, a nasty change from what we were quite good at handling, and our time is limited to handle this new torrent of, well, crap.
Some will have noticed we're still hovering just under 9 million blogs on our count, while Technorati now proudly proclaims to have 55 million. Piffle. Our model is different, however, we remove old/dead blogs from the index, plus our hardware restrictions (no VC funding for this little group, remember) means we have to ignore some legitimate blog groups (LiveJournal) that would otherwise swamp us. I'd say our spam filtering is a little better so far also - at least, it is while we dont use the blo.gs feed.
What to do now...
Over the last few years, the Blogshares index has grown, to proportions we wouldn't have expected those few years ago, back when we were at under a million, and C's goal was 5 million blogs. Here's what our growth looks like over the last two years (click for larger version):
In a few places, one of them very recently, more blogs were removed than added, from a combination of a short time where we purposefully didn't add new blogs, and a cleaning of the index to remove the splog filth. You'll notice a stronger rise over the last year, we can attribute this to the rise of MSN Spaces more than any other service.