Today, we’re releasing Insights to help you better understand and iterate upon the performance of your Wit app. As a first step we are launching a tool within Insights that makes it easier to set the confidence level for your Wit entities.
Over the past year we have received great feedback from our community, and this release is the next step in our efforts to make our NLP API more accurate, reliable and scalable for everyone.
You will now find a new Insights section on the entity page in the Wit console. If you have validated enough data with your bot, this section will show how your Wit models perform at different confidence thresholds. To start, you will be able to measure precision and recall for your Wit entities, allowing you to assess the confidence levels for both the accuracy and frequency at which you are able to identify an entity.
This allows you to pick the confidence threshold depending on what you want to optimize. More info here.
This is just the beginning, we will be adding more Insights in the future to help you better understand your app’s performance and improve its quality. As always, feel free to reach out if you have any questions, comments, or suggestions.
We have a few updates to share with you today - most exciting is the launch of Built-in NLP for Messenger, launching with Messenger Platform 2.1 just released today.
By testing and learning from our Bot Engine beta, we determined that it made the most sense to refocus on pure NLP to make it accurate, reliable and scalable for everybody. As a result, we’re sunsetting Bot Engine and will deprecate the Stories UI for new apps starting today. We will also stop serving requests to the associated
/converse endpoint on February 1, 2018.
Over the past year, we have received a lot of questions and feedback on how to integrate Wit.ai NLP into bots for Messenger. Currently, if you are leveraging an NLP API, it is an additional layer that adds both latency and complexity. We believe almost every bot should use NLP in some way to create delightful experiences. Today, we are excited to make this easier by integrating Wit directly into the Send/Receive Messenger API.
When Built-in NLP is enabled, it automatically detects meaning and information in the text of messages that a user sends, before it gets passed to the bot. This first version can detect the following entities: hello, bye, thanks, date & time, location, amount of money, phone number, email and a URL. This is the first step in bringing NLP capabilities to all developers. Learn more.
Sunsetting Bot Engine
Most of you may know that we created Bot Engine at the beginning of 2016 to help developers build text-based conversational bots. At that time, NLP was still new to many, the bot ecosystem was small and developers did not have the tools we have today (e.g. Chatfuel, Botpress, etc.).
Bot Engine was designed for complex text-only transactions because messaging platforms did not have GUI (Graphical User Interface, like webviews) elements then. The idea was to replace forms with conversations. However for many use cases, a dialog does not provide the pleasant user experience you get with a GUI (constant visual feedback, ability to modify previous choices, etc.).
That’s why in the last year, the ecosystem has shifted towards a mix of NLP and GUI elements to produce a user experience on par with native and web apps. For instance, Messenger has introduced quick replies, menus and even web view. As a result, Bot Engine and its emphasis on text-only bots has become somewhat obsolete.
Upon reviewing the top apps using Stories, we’ve mainly seen apps using 1-turn stories. These FAQ-like apps can easily be achieved with our NLP endpoint (
/message) because it’s about understanding the question and then mapping the user intent to an answer that can be stored outside of Wit. Example here. These apps don’t need the power and complexity of Stories (having to do 4 calls on average to
/converse to get to an answer as opposed to 1 single call for
/message). The recommended migration is to use
/message and code on your side to manage the conversation and link to the answers that would be stored in your database.
The numbers talk for themselves. Since the launch of Bot Engine in 2016, we have seen our community grow from 20,000 to more than 100k developers. Most of them build bots for Messenger, Slack, Telegram, and other platforms and use our NLP API (
/message). Currently, more than 90% of the Wit API calls are coming from our NLP API.
This is why we are deprecating Bot Engine and the Stories UI today, and will stop serving requests to the associated
/converse endpoint on February 1st 2018 to give developers enough time to migrate affected apps. Please check our GitHub tutorial for more details.
We want to make NLP work really well, at scale. This lets developers focus on building the best possible user experience, be it conversational, or hybrid NLP/GUI. This is why we have scaled our
/message endpoint to make it easy to programmatically interact with Wit.
Moving forward, we are working on:
- improving the quality of our NLP by leveraging cutting-edge algorithms developed at Facebook
- making it easy to share, reuse, and collaborate on entities from the community
- helping other platforms leverage our NLP API behind the scenes
As always, feel free to reach out if you have any questions, comments, or suggestions.
Wit relies both on machine learning and rules-based systems. One of these rule-based systems is Duckling, our open-sourced probabilistic parser to detect entities like dates and times, numbers, and durations. Due to our extensive growth, Duckling was not scaling as fast as we were. After considering several options, we decided to go with a rewrite in Haskell.
We were looking for a performant language with a strong support within Facebook. Two natural candidates emerged: C++ and Haskell.
A strength of Duckling is the simple domain-specific language (DSL) we provide to write rules. It allows us to get amazing open-source contributions in languages that we don’t speak. Haskell is notoriously one of the best choices out there to provide a type-safe DSL, whereas C++ would have required ninja coding skills (which we don’t have) to do the same without segfaults!
Haskell is highly scalable (it powers Facebook’s Sigma service, serving millions of requests per second) and offers type safety. Haskell is a purely functional and very expressive language. This makes it the perfect choice for Duckling!
Our main purpose in rewriting Duckling was scalability. We have been working a lot on improving the core engine algorithms as well as the time module internals, which is by far our most complex component. There is still plenty of room for improvement, but Duckling is more efficient than ever. As an example, Duckling is now used at scale internally by Facebook.
Due to Haskell’s strong type system, we had to get rid of our free-form Clojure maps. The end result is a more structured and readable codebase.
We have started housing language-agnostic rules under a common umbrella. As a result, each new language gets a basic support for amounts of money, emails, phone numbers and URLs for free.
In order to get us up and running quickly on Haskell, we are deprecating the Clojure library today and we will commit those resources into making the Haskell version word class. The new Duckling is up-to-date with the latest Clojure code.
Because we want to make sure the transition is smooth, we are providing a standalone HTTP server to get Duckling results as JSON.
Since we’ve open-sourced Duckling two years ago, our Wit community has contributed in many languages and driven real world usage, enhancing Duckling’s capabilities. Thank you to all our contributors who help make Duckling more robust and polyglot for everyone!
Duckling is now available on GitHub.
You can start diving into the new codebase today. To give you a head start:
# Clone the repository
$ git clone https://github.com/facebookincubator/duckling.git && cd duckling
# Install Stack
$ curl -sSL https://get.haskellstack.org/ | sh
# Compile and run the example
$ stack build && stack exec duckling-example-exe
# Run the tests
$ stack test
We are looking forward to see how you will use it, help extend the rulesets to make Duckling more robust, and teach it new languages.
Today, we’re releasing in Beta 3 new built-in entities that many of you have requested:
These binary entities are of type trait and will help you detect when people say hello, bye or thanks to your app!
Why trait? As you probably know, trait entities are entities that are spanless. Like the classic “intent” entity, trait entities are determined by the sentence as a whole rather than one or certain other words in the sentence. For example: “I’ll see you later”, “see you later”, “bye bye” and “goodbye” are examples of the wit/bye entity.
Why binary? They only have one value that’s “true”. So either the sentence has this meaning and the value=true or the sentence doesn’t have this entity. We’ve always wanted to do multiple intent detection and now binary trait entities allow you to detect multiple intents in a query.
We hope you’ll like these new entities. Since they still need a bit of training, they’re being released in Beta for English. Use them in lieu of your custom intents. As with any built-in entities, they’ll be leveraging all the Wit apps that use them and will be much more inclusive and accurate over time.
As always, feel free to reach out if you have any questions, comments, or suggestions.
In September 2013, we created Wit to make it easy for developers to add a natural language interface to their apps or devices. You can read more on our journey here. For the past 3.5 years, our community has kept growing. Today, we are pleased to announce that we’ve passed 100,000 developers in the Wit community building mobile apps, devices, robots, and bots for Messenger, Slack, Telegram, and other platforms!
To cope with this outstanding growth (from 20,000 developers 12 months ago), we have been working a lot behind the scenes on scalability and stability by leveraging the Facebook stack. Now we are ready to scale and we will start improving our NLU algorithms by building on top of Facebook’s AI and NLP platforms.
We have learned a ton since the launch of Bot Engine last year. Bot Engine has an ambitious goal to learn conversation flows from examples. It is still in beta until it meets that goal. As an alternative, most of you have been using natural language understanding to extract structured data first, and then rely on custom code or other bot platforms to handle longer conversations.
Speaking of bot platforms, since launching Wit.ai 3 years ago, we’ve been focused on arming developers with a solid API to build great experiences for their users. We’ve been heavily reliant on our web app to manage certain steps, such as creating a new app or validating expressions with several entities. Because of the strong network of developers that are a part of Wit.ai, we’ve seen more and more platforms being built on our API especially in the last 12 months… with the rise of chatbots. These platforms have been building their own web interface tailored for their use cases. In true hacking fashion, these platforms had to be very creative (i.e. reverse engineer Wit) to use the API behind the scenes.
Today we are updating our Wit.ai API to make it even easier for developers and platforms to understand natural language at scale.
POST /apps to create a new app
PUT /apps/:id to update the language or timezone of an existing app
POST /samples to validate expressions with multiple entities (trait and or keyword/free-text)
We are also reintroducing the N-Best feature. Simply put, it means that now Wit.ai will not just return the best intent, but a ranked list of several possible intents. This is especially helpful to implement a reformulation strategy if you’re implementing an FAQ: for an ambiguous query, your app could say “Did you mean X,Y or Z…”. It is also great to implement your own training mechanism: when Wit is not super confident, you could retrieve the 3 most relevant entities and let your user validate the correct one…
Please look at the HTTP API reference for more details. You can also check out the new github tutorial
This is indeed a first step, but if you’re thinking of building a platform on Wit, this should get you started. If you run across issues, please drop us a line!
Finally, we wanted to thank you for being a part of our community. Your energy, continuous enthusiasm and endless support have been a great source of motivation in this journey.