In my thesis work I'm currently using Node.js (with Express etc, etc) as a client for doing web search with the Bing Search API under the hood. This works fine, it is snappy and a delight to work with.
However, under the hood there is more than meets the eye. Bing Search API has a "page size" limitation of 50, and a overall search limit of 1000 (searching through the API will give 1000 results as a maximum). This means that to get the 1000 results, 20 searches á 50 results is necessary.
However, under the hood there is more than meets the eye. Bing Search API has a "page size" limitation of 50, and a overall search limit of 1000 (searching through the API will give 1000 results as a maximum). This means that to get the 1000 results, 20 searches á 50 results is necessary.
This is not entirely straight forward due to Node.js' asynchronous nature. Some plumbing is needed to be sure that all 20 search results have arrived before presenting this to the user. There are ways around this (like showing the first 50, and putting the rest in storage when they arrive), but I wanted to get all results at the same time. So some synchronisation was needed.
As a part of testing the API, a set of offline (batch) routines was devised, and I decided to use Go for this. Using a language for this designed for concurrency must be smart, yes? Only there wasn't a full Bing API for Go.
Nice reading, I love your content. This is really a fantastic and informative post. Keep it up and if you are looking for Bing Search Api then SERP House
ReplyDelete