The Ape Machine
SIGN UP FOR OUR NEWSLETTER
Over the years I have many times ran into the problem of extracting keywords, especially from short text volumes like single sentences, titles, or questions.
It has been a long known problem to deal with short text in NLP, and identifying keywords is particularly difficult to do.
I have tried many approaches over the years, from my own very naive implementation of a simple scoring algorithm, based on a very popular word game, to Rapid Automated Keyword Extraction (RAKE), none of which performed very well if there wasn't a large document to work with.
Eventually, as I grew more familiar with common machine learning models in the modern age, I started seeing keyword extraction as a "simple! translation problem, and in this article we will go over the journey that led to a great keyword extractor that we can now use in production.
By adding chatbot capabilties to a command line interface we will be able to experiement with a more natural language approach to dealing with the console. This is mostly a toy exercise, and a way to experiment with the various chatbot api platforms that are out there, but with enough work, I think something quite interesting can be done here.