Proactive annotations

16 Nov 2017 entities, feature

As part of our ongoing efforts to make Wit.ai easier and more pleasant to use, we are launching a new version of our sentence annotation widget in the web console.

When you type a new sentence in the Understanding tab to teach a new app or improve an existing one, you will now see that Wit will automatically:

  • search for existing entities as you type
  • suggest some built-in entities that may be relevant to your app.

Starting today, Wit will suggest the following entities when relevant: wit/datetime, wit/duration, wit/distance, wit/temperature, wit/quantity, wit/url, wit/amount_of_money, wit/phone_number, wit/volume and wit/email. These entities are governed by our open source linguistic probabilistic parser. Don’t hesitate to contribute to improve existing languages and locales or implement the ones you need.

When an entity is suggested, just click the checkmark to include it before validating the sentence.

As always, please reach out if you have any questions or comments. We welcome and value your feedback to make Wit better!

Team Wit



Messenger Built-In NLP

07 Nov 2017 messenger, built-in-nlp

We are excited to announce 3 updates to Built-In NLP. A few months ago, we released Built-in NLP with the Messenger Platform 2.1 to make it easier for developers to leverage the Wit NLP API when building for Messenger. Built-In NLP allows developers to get default entities (thanks, bye, dates and times, etc.) and custom entities by connecting their Wit app to their FB page without any additional call to Wit.ai!

Default Built-In NLP now in 11 languages

In addition to English, we are opening the default Built-in NLP to 10 new languages: Chinese, French, German, Dutch, Italian, Polish, Portuguese, Romanian, Spanish and Vietnamese. As a reminder, some of these entities are powered by Duckling, our open-source linguistic parser. Don’t hesitate if you want to contribute/help here!

Adding Timezone/Locale support

Following our recent support for locales in the API, we are also adding support for timezones and locales within Built-in NLP. This is very important if you rely on our date/time entity detection, powered by Duckling. If a user sends “I want to schedule an appointment in 3 hours” to your page, Built-In NLP will now be able to detect and resolve “in 3 hours” into the user timezone/locale.

Adding Advanced Settings

When you want to get custom entities, you can link your Wit app with the page via your Wit Access token. We received some feedback that you would like to also access some advanced features from the Wit.ai API. As a result, we are adding an Advanced Settings option (accessible via the API and the Facebook App UI) to allow you to:

  • Enable the Verbose option, which allows you to get extra information like the position of the detected entity in the query.
  • Enable N-best, which allows Wit to return the n-best trait entities as opposed to only the top one, by default.

More info on this new version here.

As always, please reach out if you have any questions, comments, or suggestions.

Team Wit



Locale support for dates and times

23 Oct 2017 context, duckling

Today, we are adding locale support to improve detection and resolving of date and time expressions.

Context is key in NLP. This is especially true when trying to understand dates or locations. For instance, “today” will be interpreted as differently depending on the timezone of the user: the same absolute time may be the 17th in San Francisco and the 18th in Hong-Kong.

You can already pass reference_time or timezone as context to help resolve dates and times. Starting today you can also provide a locale.

Depending on what country your user is from, the same sentence may mean different things. “remind me on 11/12” means Nov 12th in the US but Dec 11th for our mates in the UK! Similarly, “Thanksgiving 2017” means November 23rd in the US, while it means October 9th to a Canadian.

If the user locale is supported by Duckling, our open-source linguistic parser, Wit will use it to deliver the correct result for your user. If a locale is not yet available in Duckling, it will default to the “parent” language with no locale-specific customizations. We welcome contributions from the community to help us improve our coverage of various locales. Head over to our GitHub repository to contribute!

Here is an example of calling Wit passing the “en_GB” locale:

curl \
-H 'Authorization: Bearer $WIT_ACCESS_TOKEN' \
'https://api.wit.ai/message?v=20171018&q=remind%20me%20on%2011%2F12&context=%7B%22locale%22%3A%22en_GB%22%7D'

{"msg_id":"0Wo02v2lMNlNtmSQW","_text":"remind me on 11\/12","entities":{"datetime":[{"confidence":0.9668475,"values":[{"value":"2017-12-11T00:00:00.000-08:00","grain":"day","type":"value"},{"value":"2018-12-11T00:00:00.000-08:00","grain":"day","type":"value"},{"value":"2019-12-11T00:00:00.000-08:00","grain":"day","type":"value"}],"value":"2017-12-11T00:00:00.000-08:00","grain":"day","type":"value"}]}}

As always, feel free to reach out if you have any questions, comments, or suggestions!

Team Wit



Clearer controls over entity lookup strategies

23 Aug 2017 entities, news

Today we are making it easier for you to know how Wit finds entities in a sentence. Wit allows you to extract meaning from a user query as a list of entities. We employ various lookup strategies for entities, as described in our docs. Wit can use the following strategies to find entities in sentences:

  • predetermined keywords (keywords strategy)
  • patterns and positions in the sentence where certain words often appear (free-text strategy)
  • common characteristics of the sentence as a whole (trait strategy)

You usually don’t have to worry about lookup strategies, but sometimes it can come in handy to understand and tweak how Wit detects entities.

To keep Wit’s core experience robust and simple, we made these updates:

  • When you first validate a sample (either in Console or via the POST /samples API), Wit finds the right lookup for your entity: keyword, free-text, keyword & free-text or trait without you having to select it yourself.
  • If an entity is being misused (i.e. forgetting to highlight a keyword), Wit detects this and alerts you at validation time.
  • You can change the lookup strategy for a given entity in the Understanding tab, in a more intuitive way.
  • If you are about to update a lookup strategy, you will be prompted to confirm your decision, ensuring that you didn’t make the update unintentionally.

We’re happy to make these updates based on the feedback we’ve received from our community. Thank you for your support, and as always, we look forward to hearing how we can continue to improve your experience with Wit.

Team Wit



Introducing Insights, a way to better understand your app performance

08 Aug 2017 entities, news

Today, we’re releasing Insights to help you better understand and iterate upon the performance of your Wit app. As a first step we are launching a tool within Insights that makes it easier to set the confidence level for your Wit entities.

Over the past year we have received great feedback from our community, and this release is the next step in our efforts to make our NLP API more accurate, reliable and scalable for everyone.

You will now find a new Insights section on the entity page in the Wit console. If you have validated enough data with your bot, this section will show how your Wit models perform at different confidence thresholds. To start, you will be able to measure precision and recall for your Wit entities, allowing you to assess the confidence levels for both the accuracy and frequency at which you are able to identify an entity.

This allows you to pick the confidence threshold depending on what you want to optimize. More info here.

This is just the beginning, we will be adding more Insights in the future to help you better understand your app’s performance and improve its quality. As always, feel free to reach out if you have any questions, comments, or suggestions.

Team Wit