After the Deadline

Three Steps to Perfectly Proofed Posts

Posted in HOWTO by Michelle W. on May 16, 2013

Tools like After the Deadline (and the proofreading tools built in to WordPress.com) are great for catching typos and making sure that an overlooked mis-type doesn’t cloud your message. As anyone who’s ever relied on a proofreading tool knows, however, there’s more to clean copy than a lack of typos.

Use After the Deadline as your personal proofreading assistant, but consider the other key copyediting steps before you hit “publish.” Rely on this copyediting trifecta to give your sparkling prose a final polish:

  1. Accuracy. Is everything you’ve written correct? This is critical if what you’re writing includes things like dates, contact information, or data — it’s easy for our eyes to skip over a street addresses or URL we’ve seen a hundred times, so make a point of double-checking. Accuracy is about more that just verifying statistics and email addresses, though. If you’re giving instructions, are they clear? Is there anything you’re written that can be misinterpreted? Be your own devil’s advocate, and give your writing a once-over for accuracy; your piece will be stronger for it.
  2. Nips and Tucks. Editing is just as important as writing; the first few drafts you write are just that — drafts. Once you’ve got all your ideas down clearly and accurately and you think you’re happy with what you’ve got, take a figurative x-acto knife to your words. Is each word necessary? Is each word as strong as it can be? Does each word advance your point? Be merciless.
  3. Spelling and Grammar. Here’s where After the Deadline comes in! Do a final sweep of your tight and toned piece to eliminate distracting typos and grammar gaffes. Even your most lyrical description will land with a thunk if your “two” should have been a “too.”

If you’ve got a self-hosted WordPress blog, the multi-functional Jetpack plugin gives you After the Deadline along with dozens of other goodies, and there are After the Deadline extensions available for Chrome, Firefox, and Open Office. Whatever your site’s platform, there’s a tool to help you look your best.

Tagged with: ,

Have you installed Jetpack yet?

Posted in HOWTO by Michelle W. on February 20, 2013

In our last post, we let you know that we’d no longer be supporting After the Deadline browser extensions — you can still use AtD, just within WordPress itself rather than through a browser.  If your site is on WordPress.com, you’re all set; you’ve got a “proofread” button right in your post and page editors.

But what if you’re prone to typos and running a self-hosted WordPress blog? Easy: Jetpack.

Jetpack is the Swiss army knife of plugins: it’s a multipurpose plugin that gives your self-hosted WordPress site two dozen of the most popular features of WordPress.com. By hooking your site into the WordPress.com cloud, it lets you have the best of both worlds: WordPress.com power and ease with WordPress.org flexibility.

Why are we going on about Jetpack on the After the Deadline blog? Simple: installing Jetpack gives you AtD, right there in your post editor, so you can check your work no matter where you’re publishing. And if that isn’t enough, consider some of the other features it includes:

  • Tiled photo galleries, so you can create elegant, magazine-style mosaic layouts with a few clicks.
  • Shortcode embeds to speed up the process of inserting multimedia.
  • Comments that let readers respond to you using their WordPress.com, Facebook, or Twitter login.
  • Notifications and stats to help you manage everything from one simple toolbar.
  • Sharing tools to take your content from stand-alone post to viral sensation.

That’s just the tip of the jetpack – check out Jetpack.me for a full features list. Best of all, this can all be yours for the low, low price of zero dollars, and with only one plugin to install.

Remember, if you still want to pitch in with the development and maintenance of browser-specific After the Deadline extensions, the main AtD site has tons of developer resources. But if you just want to make sure you never publish another “teh” (at least, accidentally), then Jetpack’s your plugin.

Tagged with: , ,

Add Grammar and Spell Check to Any WYSIWYG Editor

Posted in HOWTO by rsmudge on January 5, 2010

After the Deadline front-end libraries are available for jQuery and TinyMCE. You’ve asked what it would take to make AtD available in another WYSIWYG environment or even in a web-based word processor. In the past I’d answer that you’d need to study the code to either the jQuery or TinyMCE extensions, visit the mountains, meditate and wait for the answer to come.

Today I’m bringing the mountain to you. Much of the code that makes After the Deadline work in TinyMCE and jQuery is similar. Painfully similar. To the point where a bug in one is a bug in the other. I’ve refactored these extensions and created an After the Deadline Core UI module. This module is browser independent with no external dependencies. It provides functionality to parse the AtD XML into an error data structure, retrieve suggestions and other information given an error phrase and a word of context, and it also abstracts away the logic to traverse a DOM and insert the AtD markup for errors.

To test this module, I gave myself one working day to port After the Deadline to CKEditor. CKEditor is a WYSIWYG editor, similar to TinyMCE. I have never used CKEditor before and this was a challenge. Still, I was able to make it quite far and in this post, I’ll show you how to add AtD to an editor using the Core UI Module.

Here is what an After the Deadline editor plugin must do:

  1. The first step is to setup the AtD Core UI module:
    atd_core = new AtDCore();
  2. The next step is to define several functions that the AtD Core UI module expects. The module will use these functions to manipulate the DOM and find elements in the way that your environment expects. The full list of functions is documented in the AtD Core UI README. Here are a few of these functions from CKEditor:
    atd_core.replaceWith = function(old_node, new_node) {
       return new_node.replace(CKEDITOR.dom.element.get(old_node));
    };
    
    atd_core.create = function(node_html) {
       return CKEDITOR.dom.element.createFromHtml( '<span class="mceItemHidden">' + node_html + '</span>' );
    };
    
  3. Once these functions are defined you can set AtD-specific preferences like the list of strings to ignore and which types of errors to show.
    atd_core.showTypes('Complex Expression, Diacritical Marks, Double Negatives, Redundant Expression');
    atd_core.setIgnoreStrings('CKEditor, was thrown');
    
  4. When a user requests proofreading, it is up to you to extract the contents of the editor and post it to the After the Deadline service. Here is how I extract the editor contents in CKEditor:
    var editor_contents = editor.document.getBody().getHtml();
    
  5. You should receive an XML document from the server with an error message or a data structure containing the AtD data. To check the XML document for an AtD error:
    function ajax_callback(xml_response) {
       if (atd_core.hasErrorMessage(xml_response)) {
          alert(atd_core.getErrorMessage(xml_response));
          return;
       }
    
  6. If there are no errors then you’ll want to parse the XML into a data structure the Shared UI code can use. Use the processXML function to do this. It will return a JavaScript object that you will be using again.
    var results = atd_core.processXML(xml_response);
  7. Now, let’s say you want to highlight errors in your editor. Great! Extract the contents of the editor (should be an array of elements from the root element) and pass these to markMyWords. Using the prototypes you provided earlier, this function will walk through these nodes and highlight the errors. Be thankful that you didn’t have to write the code to do this.
    var nodes = editor.document.getDocumentElement().getChildren().getItem(1)['$'].childNodes;
    atd_core.markMyWords(nodes, results.errors);
    
  8. Earlier, I hope you attached a click or context menu listener to your editor. If you did you can use isMarkedElement on the event target when a click occurs in your editor. If this returns true, this is your clue that a user clicked on a marked error and you should display a menu offering them suggestions.To get the suggestions, call findSuggestion using the marked element. This will return a JavaScript object with the following members that may interest you: suggestions, description, moreinfo
    editor.contextMenu.addListener(function(element) {
       if (atd_core.isMarkedNode(element.$)) {
          var meta = atd_core.findSuggestion(element.$);
          var commands = {};
    
          addItem(editor, meta.description, function() { }, 0, commands, 'AtD_description');
    
          for (var x = 0; x < meta.suggestions.length; x++)
             addItem(editor, meta.suggestions[x], makeCallback(element.$, meta.suggestions[x]), x + 1, commands, 'AtD_suggestions');
    
          addItem(editor, 'Ignore', makeIgnoreCallback(element.$, element.$.innerHTML), 1, commands, 'AtD_ignore');
          addItem(editor, 'Ignore All', makeIgnoreAllCallback(element.$, element.$.innerHTML), 0, commands, 'AtD_ignore');
    
          return commands;
       }
    });
    
  9. For each suggestion, I use makeCallback to generate a function to attach to the menu item. This generated function calls applySuggestion in the Core UI Module. You should use applySuggestion as it’s smart enough to do what the suggestion asks. For example, the (omit) suggestion removes the word in question:
    var makeCallback = function(element, suggestion) {
       return function() {
          atd_core.applySuggestion(element, suggestion);
       };
    };
    
  10. You may want to add functionality to let the user ignore the current error or to ignore all occurrences of it. The “Ignore suggestion” menu item removes the marked node keeping its children. No need to call into the AtD Core UI module:
    var makeIgnoreCallback = function(element, word) {
       return function() {
          CKEDITOR.dom.element.get(element).remove(true);
    };
    

    To ignore all occurrences of an error, you can use the removeWords function in the Core UI Module.

    var makeIgnoreAllCallback = function(element, word) {
       return function() {
          atd_core.removeWords(undefined, word);
       };
    };
    
  11. The last thing you should do is attach a listener to remove the After the Deadline markup when the contents of the editor are grabbed. You can remove the AtD markup using the removeWords function.

And that’s it. Add in the necessary scaffolding for your editor and you have After the Deadline integration. Here are some other things you can do:

  • Make an Ignore Always menu option save the user’s preference in a cookie or on your server. Resurrect this setting later with the setIgnoreStrings function.
  • Keep track of the user’s preference for what types of errors to show. Use showTypes to enable these for the user.

These are the kinds of things we do in WordPress to make AtD a first-class feature for our users. If you’re looking to port After the Deadline to another environment, the AtD Core UI module will save you a lot of time.

Coming up I’ll be releasing versions of the AtD/jQuery and AtD/TinyMCE Extensions using this shared module. The AtD/CKEditor extension is available now.

Add After the Deadline to Your Application

Posted in HOWTO by rsmudge on November 12, 2009

1. Why After the Deadline

Welcome to Raffi’s practical guide to adding After the Deadline to your application. After the Deadline is a technology that checks spelling, style, and grammar. Through it you can bring word processor quality writing tools to the users of your application. And it’s open source. How much better can it get?

2. Server Software

After the Deadline uses a client-server architecture. Your program embeds code that talks to an After the Deadline server. When the user clicks the spellcheck button their document is sent to a server, processed, and the errors are sent back. This code in your application then proceeds to highlight these errors and present them to the user.

After the Deadline architecture

To add After the Deadline to your application you will need to run an After the Deadline server. You can download the code for the server from:

http://open.afterthedeadline.com/download/download-source-code/

You only need the source code package and don’t worry–precompiled binaries are distributed in the package.

To run After the Deadline you’ll need Java 1.6.0 and 4GB of memory. You can put it on a virtual machine so long as it has enough memory. After the Deadline will happily use as many CPU cores as you can give it. If you’re not processing a lot of requests you’ll only need a single core. We run our After the Deadline server on an eight core machine but we have a lot of users, your needs may vary.

The memory requirements are steep but this is because After the Deadline loads its language models into memory. After the Deadline uses these models to make smart spelling corrections, decide if a word is misused, and to avoid bogus grammar and style errors.

Installation of After the Deadline is pretty easy. Extract the archive where ever you like and execute ./run.sh to start the server. We’ve run the After the Deadline server on Linux and MacOS X

Screenshot of After the Deadline running.

After the Deadline will listen on port 1049 of your localhost. For security reasons it only accepts connections from 127.0.0.1. We use nginx to proxy connections from port 80 to 127.0.0.1:1049. You can do this or if you’re feeling adventurous you can edit run.sh and change -Dbind.interface=127.0.0.1 to reflect the external IP address of your server.

Regardless of which choice you make, you’ll want to take note of the IP address and port number of your After the Deadline server. We will use this information later.

3. Talking to the Server

As a security measure, JavaScript uses a same origin policy for AJAX calls. JavaScript executed on a webpage can only send requests to the server the script was downloaded from. It’s quite understandable that you’ll want to run After the Deadline on hardware that is separate from your web server. This poses a problem as this policy prevents any front-end integration from talking to your After the Deadline server.

The solution to this problem is to create a proxy script on your webserver that receives AJAX calls and sends them to your After the Deadline server. In our TinyMCE and jQuery packages we include this proxy.php file that you can modify to your needs.

$API_KEY = "";

if ( $_SERVER['REQUEST_METHOD'] === 'POST' )
{
 $postText = trim(file_get_contents('php://input'));
}

if (strcmp($API_KEY, "") != 0)
{
 $postText .= '&key=' . $API_KEY;
}

$url = $_GET['url'];

/* this function directly from akismet.php by Matt Mullenweg.  *props* */
function AtD_http_post($request, $host, $path, $port = 80)
{
 $http_request  = "POST $path HTTP/1.0\r\n";
 $http_request .= "Host: $host\r\n";
 $http_request .= "Content-Type: application/x-www-form-urlencoded\r\n";
 $http_request .= "Content-Length: " . strlen($request) . "\r\n";
 $http_request .= "User-Agent: AtD/0.1\r\n";
 $http_request .= "\r\n";
 $http_request .= $request;

 $response = '';
 if( false != ( $fs = @fsockopen($host, $port, $errno, $errstr, 10) ) )
 {
   fwrite($fs, $http_request);

   while ( !feof($fs) )
   {
     $response .= fgets($fs);
   }
   fclose($fs);
   $response = explode("\r\n\r\n", $response, 2);
 }
 return $response;
}

$data = AtD_http_post($postText, "service.afterthedeadline.com", $url);

header("Content-Type: text/xml");
echo $data[1];
?>

To adapt this script, change service.afterthedeadline.com in line 42 to the address of your server. I don’t recommend hardcoding an API key as the After the Deadline server caches information about your writing and uses the key to lock this data structure. If every request uses the same key, you will unnecessarily bottleneck those requests.

If your After the Deadline server receives requests on a port other than port 80, you may append this port to the AtD_http_post function after $url:

$data = AtD_http_post($postText, "192.168.1.101", $url, 1049);

If you don’t use PHP on your server then you will need to create your own AJAX proxy script. This is a common problem and several solutions are available for most languages.

Now that you have a proxy script, note this URL as we will use it when we integrate After the Deadline with your application’s front-end.

4. Front-End Integration

Now that you have a server and a means to communicate with it, the last step is to integrate After the Deadline into the front-end of your web application. Several options are available at http://www.afterthedeadline.com/development.slp. Here I will focus on the After the Deadline extension for jQuery. jQuery is a JavaScript library that removes the pain of working with multiple browsers.

Download the After the Deadline package for jQuery and extract it on your webserver. You can get it from: http://www.afterthedeadline.com/download.slp?platform=jQuery

Adding After the Deadline to an application involves:

  1. loading the right resources
  2. creating your form
  3. telling the After the Deadline jQuery package where to find your proxy script
  4. creating functions to check the right form element and restore the form before submitting.

Our example application is a car review website. Ok, it’s not much of an application but it illustrates these steps. It contains a TEXTAREA where the user is expected to write their review. It also has a Check Spelling link that the user can click to check their writing.

Here is a screenshot of this application in action:

broken

Here is the code to the application:

<html>
 <head>
 <title>Write a review of your car</title>

 <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js"></script>
 <script src="scripts/jquery.atd.js"></script>
 <script src="scripts/csshttprequest.js"></script>
 <link rel="stylesheet" type="text/css" media="screen" href="css/atd.css" />
 <script src="scripts/jquery.atd.textarea.js"></script>

 <script>
 function check()
 {
    AtD.checkTextArea('review', 'checkLink', 'Edit Text');
 }

 function submit()
 {
    AtD.restoreTextArea('review');
    jQuery('#reviewForm').submit();
 }

 AtD.rpc = 'server/proxy.php?url=';
 AtD.api_key = ( new Date() ).getMinutes();
 </script>

 </head>
 <body>

 <h1>Review your car!</h1>

 <p>Please write a review of your car so others may learn.</p>

 <form id="reviewForm">
   <textarea id="review" name="review" class="input" style="width: 400px; height: 100px; border: 1px solid black"></textarea>
   <p><a href="javascript:submit()">Submit Review</a> | <a href="javascript:check()" id="checkLink">Check Spelling</a></p>
 </form>
 </body>
</html>

Lines 7-9 load the After the Deadline resources. You can change the look of the After the Deadline markup by modifying css/atd.css. Line 5 loads jQuery. You want to make sure you do this first.

Lines 34-37 contain the car review form. Line 35 has the TEXTAREA. Note that the value of its id is review. I have also embedded some style information into the TEXTAREA element. When you click the check spelling button, the After the Deadline jQuery extension will do its best to mimic the look of the original TEXTAREA. You can put this information inline or assign a class to the TEXTAREA. If you want the spellchecker mode to pick up the style information, make sure it works with a DIV as well as a TEXTAREA.

Lines 23 and 24 tell the After the Deadline jQuery extension where to find your proxy script. You’ll notice I set the key to a random value.  I did this because you’re using your own server. Despite this, the After the Deadline jQuery extension expects an API key when talking to a proxy script.

In line 36 you’ll notice that Submit Link is set to call the JavaScript submit function. This function is defined on lines 17-21. In line 19 I call a function to restore your TEXTAREA before submitting the form. This is important as After the Deadline replaces the TEXTAREA with a DIV during proofreading. This function does nothing if the TEXTAREA doesn’t need to be restored.

Line 36 also has a Check Spelling link that calls a JavaScript check function. This function is defined on lines 12-15. On line 14 I call an After the Deadline jQuery extension function to check the TEXTAREA. The first parameter is the id of the TEXTAREA. The second parameter is the id of the check spelling link. Within this link you can use any HTML you like. The third parameter is what the link should say when After the Deadline is checking the TEXTAREA. This is a good place to tell the user that their text is not editable.

You can try this example at http://www.polishmywriting.com/atd_jquery/carreview.html.

5. Can I use After the Deadline with my application?

The After the Deadline server software is released under the GNU General Public License (GPL). You’re welcome to use and redistribute this server software even for commercial purposes. If you choose to redistribute the After the Deadline server with your changes then you must make the source code with your changes available under the GNU General Public License. This only applies to the After the Deadline server software. Since this software is a stand-alone server, the requirements of the GPL shouldn’t burden you too much.

The client side libraries like the jQuery extension are available under the GNU Lesser General Public License. This license grants you the same freedoms as the GPL while including an exception that states only changes to the library need to be redistributed with source code. You’re able to use these libraries in your closed source application.

6. The Next Step

Now you know how to add After the Deadline to your application. It’s a simple matter of installing the server, creating a proxy, and integrating with your front-end.

If you have questions, you’re encouraged to ask them on the atd-developers mailing list.

Attempting to Detect and Correct Out of Place Plurals and Singulars

Posted in HOWTO, NLP Research by rsmudge on September 22, 2009

Welcome to After the Deadline Sing-A-Long.  I’m constantly trying different experiments to catch my (and occasionally your) writing errors.  When AtD is open sourced, you can play along at home.  Read on to learn more.

Today, I’m starting a new series on this blog where I will show off experiments I’m conducting with AtD and sharing the code and ideas I use to make them happen.  I call this After the Deadline Sing-A-Long.  My hope is when I finally tar up the sourcecode and make AtD available, you can replicate some of these experiments, try your own ideas, have a discovery, and send them to me.

Detecting and Correcting Plural or Singular Words

One of the mistakes I make most is to accidentally add an ‘s’ to a word making it plural when I really wanted it to be singular.  My idea?  Why not take every plural verb and noun and convert them to their singular form and see if the singular form is a statistically better fit.

AtD can do this.  The grammar and style checker are rule-based but they also use the statistical language model to filter out false positives.  To set up this experiment, I created a rule file with:

.*/NNS::word=:singular::pivots=,:singular
.*/VBZ::word=:singular::pivots=,:singular

AtD rules each exist on their own line and consist of declarations separated by two colons.  The first declaration is a pattern that represents the phrase this rule should match.  It can consist of one or more [word pattern]/[tag pattern] sequences.  The tag pattern is the part-of-speech (think noun, verb, etc.).  After the pattern comes the named declarations.  The word= declaration is what I’m suggesting in place of the matched phrase.  Here I’m converting any plural noun or verb to a singular form.  The pivots specify the parts of the phrase that have changed to inform that statistical filtering I mentioned earlier.

The next step is to create a file with some examples to test with.  I generally do this to see if the rule does what I want it to do.  Here are two of the sentences I tried:

There are several people I never had a chance to thanks publicly.
After the Deadline is a tool to finds errors and correct them.

So, with these rules in place, here is what happened when I tested them:

atd@build:~/atd$ ./bin/testr.sh plural_to_singular.rules examples.txt
Warning: Dictionary loaded: 124264 words at dictionary.sl:50
Warning: Looking at: several|people|I = 0.003616585140061973 at testr.sl:24
Warning: Looking at: several|person|I = 1.1955063236931511E-4 at testr.sl:24
Warning: Looking at: to|thanks|publicly = 1.25339251574261E-6 at testr.sl:24
Warning: Looking at: to|thank|publicly = 1.7004358463574743E-4 at testr.sl:24
There are several people I never had a chance to thanks publicly.
There/EX are/VBP several/JJ people/NNS I/PRP never/RB had/VBD a/DT chance/NN to/TO thanks/NNS publicly/RB
   0) [REJECT] several, people -> I
        id         => 3095c361e8beeb60abebed29fe5657be
        pivots     => ,:singular
        path       => @('.*', 'NNS')
        word       => :singular
   1) [ACCEPT] to, thanks -> @('thank')
        id         => 3095c361e8beeb60abebed29fe5657be
        pivots     => ,:singular
        path       => @('.*', 'NNS')
        word       => :singular
Warning: Looking at: Deadline|is|a = 0.05030783620533642 at testr.sl:24
Warning: Looking at: Deadline|be|a = 0.00804979134152438 at testr.sl:24
Warning: Looking at: to|finds|errors = 0.0 at testr.sl:24
Warning: Looking at: to|find|errors = 0.0024240611254462076 at testr.sl:24
Warning: Looking at: finds|errors|and = 2.5553084536790455E-5 at testr.sl:24
Warning: Looking at: finds|error|and = 3.14416280096839E-4 at testr.sl:24
After the Deadline is a tool to finds errors and correct them.
After/IN the/DT Deadline/NN is/VBZ a/DT tool/NN to/TO finds/NNS errors/NNS and/CC correct/NN them/PRP
   0) [REJECT] Deadline, is -> a
        id         => 1ab5cd35b6146cbecbc31c8b2a6d8e96
        pivots     => ,:singular
        path       => @('.*', 'VBZ')
        word       => :singular
   1) [ACCEPT] to, finds -> @('find')
        id         => 3095c361e8beeb60abebed29fe5657be
        pivots     => ,:singular
        path       => @('.*', 'NNS')
        word       => :singular
   2) [ACCEPT] finds, errors -> @('error')
        id         => 3095c361e8beeb60abebed29fe5657be
        pivots     => ,:singular
        path       => @('.*', 'NNS')
        word       => :singular

And that was that.  After I ran the experiment against a more substantial amount of text, I found too many phrases were flagged incorrectly and I didn’t find many legitimate errors of this type.  If I don’t see many obvious mistakes when applying a rule against several books, blogs, and online discussions–I ignore the rule.

In this case, the experiment showed this type of rule fails.  There are options to make it better:

  • Set a directive to raise the statistical threshold
  • Try looking at more context (two words out, instead of one)
  • Look at the output and try to create rules that look for more specific situations where this error occurs
Follow

Get every new post delivered to your Inbox.

Join 289 other followers