After the Deadline

After the Deadline – Open Sourced!

Posted in News by rsmudge on October 26, 2009

After the Deadline, an intelligent checker for spelling and grammar, is now free software. The server software is available under the GNU General Public License.

We’ve created a portal at http://open.afterthedeadline.com where you can:

  • Download the GPL source code, models, and rule set powering After the Deadline
  • Learn how to add rules and find your way around the code
  • Get data from Wikipedia and Project Gutenberg to regenerate models if you need to

We’re also announcing a jQuery API for After the Deadline. Now you can add an AtD check to a DIV or TEXTAREA with little effort. This is the same API powering our Intense Debate plugin.

Our goal with this project is to help people write better, every where.  Now all the tools are available.  We look forward to seeing what you do with it.

After the Deadline is an Automattic project. Automattic contributes to WordPress.org and operates WordPress.com, the Akismet anti-spam service, the PollDaddy survey tool, Gravatar, and the Intense Debate distributed comment system.

Spell Check Comments with Intense Debate and AtD

Posted in News by rsmudge on October 8, 2009

I’ve heard several requests to make spell and grammar checking available in blog comments.  Today we’ve taken the first step towards that with an After the Deadline plugin for Intense Debate — the premier distributed comment system for websites and blogs.

commentss

Installation is a matter of visiting their plugin page and clicking activate.  Nothing else to it.  You don’t even need an AtD API key.  AtD/ID checks spelling, misused words, and some grammar issues.

If you want to see AtD/ID in action, visit the Intense Debate blog where they’re covering the story too.

http://blog.intensedebate.com/2009/10/08/after-the-deadline-intensedebate-plugin/

The AtD/ID plugin works with TypePad, Blogger, WordPress, and wherever you choose to install the ID widget.  I’ll consider this effort successful when someone publicly asks “why does my comment field have better writing correction than my publishing platform?” <G>

P.S. source code for the plugin is available and I’ll be posting an AtD/jQuery API soon.  Keep an eye out for that.

AtD Updates – Split Words and Possessive Errors

Posted in News by rsmudge on October 6, 2009

I’m back from my vacation.  Ok, I’ve been back for over a week now.  I’m working on some stuff that will make it easier to add AtD to more applications.  In the mean time, from your suggestions I’ve added some new rules to AtD.  Besides the usual tweaking here is what you get:

The spell checker now splits the misspelled word and rates these split phrases with the other suggestions.  This means typing alot, you’ll now see a lot as the top suggestion.  It also means you’ll see the right thing when you accidentally run two words together such as atleast -> at least.  I was notified about the latter error on Twitter.

Our local Media Engineer, Raanan, noted that AtD didn’t find an error in the phrase “if this is your companies way of doing support”.  If you’re curious, here is the what the process looks like:

<raanan> raffi: tried “I wonder if this is your companies way of providing support”
<raanan> AtD didn’t flag it
<raffi> I can add some grammar rules for your followed by a plural noun and try to correct it to a possessive noun
<raffi> your .*/NNS::word=your \1:base’s::pivots=\1,\1:base’s
<raffi> whether that catches some real errors or not is up for debate, can try it though
<raanan> could be interesting
<raffi> I’m trying it now
<raffi> rule development is taking an error like you gave me, going through some steps to make a rule for it, and testing it on a bunch of text I have laying around
<raffi> if it finds errors, it gets included–if it doesn’t… it goes nowhere
<raffi> or gets tweaked
<raffi> if it flags too many things as correct, I keep trying to refine it (can usually be done) until it just catches errors (although it might miss some errors to keep from flagging correctly written stuff)
<raffi> when I first made the grammar checker, I spent 3 weeks straight doing that, it got very tedious
<raffi> $ ./bin/testr.sh test.r raanan.txt
<raffi> Warning: Dictionary loaded: 124314 words at dictionary.sl:50
<raffi> Warning: Looking at: your|companies|way = 3.8133735008675426E-5 at testr.sl:24
<raffi> Warning: Looking at: your|company’s|way = 2.955364463172345E-4 at testr.sl:24
<raffi> I wonder if this is your companies way of providing support
<raffi> I/PRP wonder/VBP if/IN this/DT is/VBZ your/PRP$ companies/NNS way/NN of/IN providing/VBG support/NN
<raffi> 0) [ACCEPT] is, your companies -> @(‘your company’s’)
<raffi> id         => c17ed0984ed4d01ac172f0afd95ee00c
<raffi> pivots     => \1,\1:possessive
<raffi> path => @(‘your’, ‘.*’) @(‘.*’, ‘NNS’)
<raffi> word       => your \1:possessive
<raffi> ding
<raffi> let’s see if that works against a bunch of written text

And it did.  I tested a broader version of the rule against a big corpus of written text and found several errors with only one false positive.  Here are the final rules:

That|The|the|that|Your|your|My|my|Their|their|Her|her|His|his .*/NNS .*/NN::word= \1:possessive \2::pivots=\1,\1:possessive

If you have ideas to enhance After the Deadline or found a clear cut error that it doesn’t catch, let me know.  I’m happy to look at it.

Comments Off on AtD Updates – Split Words and Possessive Errors

Python Bindings for AtD

Posted in News by rsmudge on September 15, 2009

Miguel Ventura has been kind enough to contribute Python bindings for After the Deadline.  You can get them at:

Here is how you use them:

import ATD
ATD.setDefaultKey("your API key")
errors = ATD.checkDocument("Looking too the water. Fixing your writing typoss.")
for error in errors:
 print "%s error for: %s **%s**" % (error.type, error.precontext, error.string)
 print "some suggestions: %s" % (", ".join(error.suggestions),)

Expected output:

grammar error for: Looking **too the**
some suggestions: to the
spelling error for: writing **typoss**
some suggestions: typos

These bindings are available under an MIT license. If you’d like to contribute AtD bindings for another programming language or program visit our AtD Developer Resources page to learn more about the XML API.

Comments Off on Python Bindings for AtD

After the Deadline: Acquired

Posted in News by rsmudge on September 8, 2009

Today I have big news to announce for After the Deadline.  But first, I have to tell you a story.

I left the Air Force in March 2008 to pursue my dream of launching a startup and to finish graduate school.

Coming from the US Air Force Research Lab, I wanted to solve a problem and invent something cool.  The most recent problem I had when leaving the Air Force was writing technical reports.  I knew what I wanted to say but always had doubts about my style.  So I decided to hunker down and write a style checker.   I launched this tool as PolishMyWriting.com in July 08 and it went…. nowhere.

Later I wrote to a friend of mine in NYC who showed PolishMyWriting.com to his boss at TheLadders.com.  His boss wrote something about it in their customer newsletter and thousands of people came to my site.  They processed many documents and wrote to tell me how much this style checking tool helped them.  This inspired me.  I asked myself “if I’m selling umbrellas, where is it raining?” and I saw an opportunity in the web application space.  My goal–bring word processor quality proofreading tools to the web.  It was at this moment After the Deadline was born.

I submitted the style checker embedded into TinyMCE as part of an application to Y-Combinator, Spring 09.  Later, I was greeted with a rejection letter.  But that was ok!  I knew I didn’t need permission to start a business.  So on I went.  I adhered to the proposed schedule and milestones.

By Mar 09, I had a pretty kick ass system going.  The spellchecker accuracy was showing potential to rival even MS Word (context makes a big difference) and I knew the style checker was comparable to similar commercial software.  My favorite feature though was the misused word detection.  Outside of the latest MS Word–no one else really had this.

I applied to several other seed funds and after an encouraging meeting with a seed program in Boston, I found a trusted and qualified friend to handle business development if we were funded.  At this time I took some cost cutting measures (*cough*sold everything, moved in with my sister*cough*) to keep going.  My partner and I landed an interview and prepped like crazy for it.  We weren’t funded and I received feedback that the idea was good but the problem was too hard.  I’m thankful we had that interview though and was encouraged to make the first cut.

My short-lived partner went to a real job and I devoted another month to coding and launched in June 09.  As part of the promotion process, I posted about AtD to Hacker News. I also left this comment, hoping to impress someone:

This paragraph is from a NY Times article. Can you find the error in it?

Still Mr. Franken said the whole experience had been disconcerting. “It’s a weird thing: people are always asking me and Franni, ‘Are you okay?’ ” he said, referring to his wife. “As sort of life crises go, this is low on the totem poll. But it is weird, it’s a strange thing.”

Neither can the spell checker in your browser. Why? Because most spell checkers do not look at context. After the Deadline does.

Besides misused word detection (and contextual spell checking), After the Deadline checks grammar and style as well.

Visit http://www.polishmywriting.com/nyt.html to see the answer.

That must have worked, because later, I received an email from Matt Mullenweg asking me about bringing After the Deadline to Automattic.  Matt and I are both big believers in open source.  We like to eat but at the same time see a bigger picture where impact matters.   He is also an incredibly smooth chatter and anti-aliasing does wonders for his online presence.  I knew this was an opportunity I couldn’t say no to.

And so here I am.  We did the deal in July 09 and since then I’ve moved After the Deadline to Automattic’s infrastructure, rewrote the plugin, improved the algorithms, and today it started checking the spelling, style, and grammar for millions of bloggers.

So what’s next?

I’m continuing this natural language processing research under the Automattic banner.   We’re planning to expand AtD to support other languages.

After the Deadline will stay free for non-commercial use and we hope to see others build on the service.

And finally, our goal is to raise the quality of writing on the internet and give folks confidence in their voice.  We’re planning to open source the After the Deadline engine and the rule-sets that go with it.  This will be the most comprehensive proofreading suite available under an open source license.  I’m excited about the opportunity to be a part of this contribution.

And some thanks…

The hardest part of waiting to make this announcement is I haven’t yet had a chance to publicly thank those who believed in this project from the beginning.  I’d like to thank Mr. Elmer White for his legal counsel and support.  Congratulations on your second grandson.  Ms. Hye Yon Yi for her support and putting up with me when I was completely unavailable.  Mr. Dug Song, Patron Saint of MI Hackers, for advising me on the business side. Ms. Katrina Campau for coaching me on the investor interview.  The A2 New Tech (Ann Arbor, MI) for letting me present and being encouraging. Mr. David Groesbeck and Ms. Michelle Evanson at Why International for being my earliest business cheerleaders.  Mr. Brandon Mumby for providing hosting, even after AtD caused a hardware failure.  My colleagues, who serve at the Air Force Research Lab, for inspiring me to stay curious and keeping me in the community.  The crew in #startups on FreeNode and the Hacker News community–thanks for showing it can be done.  Of course my family (esp. my sister who let me turn the basement into a “command center”) and the makers of Mint Chocolate Chip ice-cream.

WordPress Plugin Update

Posted in News by rsmudge on September 3, 2009

Today, I’ve released an updated WordPress plugin for After the Deadline.  This plugin really revamps the AtD experience, I think you’ll like it.  [Get it here]

The most obvious change, most errors are optional and disabled by default.  AtD still checks your spelling and grammar but most of the style options are available on your profile page (/wp-admin/profile.php).

atdwporg

Here is a summary of the options:

  • Bias language may offend or alienate different groups of readers.
  • Clichés are overused phrases with little reader impact.
  • Complex phrases are words or phrases with simpler every-day alternatives.
  • A double negative is one negative phrase followed by another. The negatives cancel each other out, making the meaning hard to understand.
  • A hidden verb is a verb made into a noun. These often need extra verbs to make sense.
  • Jargon phrases are foreign words and phrases that only make sense to certain people.
  • Passive voice obscures or omits the sentence subject. Frequent use of passive voice makes your writing hard to understand.
  • Phrases to avoid are wishy-washy or indecisive phrases.
  • Redundant phrases can be shortened by removing an unneeded word.

These settings follow the user so each user on your blog can have different settings.

Also the AtD plugin code has been updated to the WordPress coding standards, much cleaner.  The TinyMCE plugin has experienced a rigorous overhaul as well.  If you have any issues, let me know.

— Raphael

(Update 4 Sept 09 – 10am) well, for those of you who had issues, thanks for letting me know. 🙂 One of the AtD files had whitespace at the end causing WordPress to die with a “Cannot modify header information” warning.  To make this especially maddening, it only affects those who have output_buffering disabled in php.ini which is dependent on who packaged PHP and/or how it was installed.   The issue has been resolved and the latest is in the WordPress repository.

It’s about time…

Posted in News by rsmudge on August 27, 2009

It’s time for After the Deadline (abbreviated AtD) to have its own blog.  For those of you who found this from a random Google search, After the Deadline is a technology that checks spelling, misused words, style, and grammar.  After a stint as a researcher for the government, this is my attempt to apply the scientific method and make good things happen.

Here I’ll keep you up to date the latest news on the technology and start sharing my results and methods.  I’m a big believer in open source and open information.  I am also frustrated by how “Computer Science” papers present lots of great ideas but often little in the way of code or data to independently verify the same results.

It’s my mission to make something immensely useful but also to share my methods and advance the science behind it.  Please join me on this journey.