What's New

We're thrilled to update you on some of the newest enhancements and give you a sneak peek into what’s coming next.

🦜🔗 ​LangChain​ Integration

Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. However, using these LLMs in isolation is often insufficient for creating a truly powerful app - the real power comes when you can combine them with other sources of computation or knowledge.

By embedding the SearchAPI, the Large Language Model can respond by querying real-time data to answer questions about current events. Its usage is particularly interesting with the "Self-ask with search" agent. This agent is equivalent to the original ​self-ask with search paper​, where a Google search API was provided as the tool.

Another common user case is summarizing long documents that naturally run into context window limitations. Here is an example ​using SearchApi to load a lengthy YouTube video transcript​, then summarize and distill it with a refine chain.

SearchApi is now natively available as part of ​langchainjs​ and ​langchain​ libraries.

For a quick overview, check out this tweet on SearchApi integration in LangChainJS.

Here is the official SearchApi documentation on its usage in Python and Javascript:
  •  Python examples​
  •  JavaScript/TypeScript examples​​

🌟 Freshly Released Engines:

  ✔️  Google Lens API​ - shipped in less than 24 hours based on ​@levelsio tweet on Twitter​. Useful as a tool in ​affiliate marketing​ for fetching product sources, links, prices, and stock information.
  ✔️  YouTube Transcripts API​
  ✔️  Amazon Search API​

🔧 Engine Improvements:

Google
  •  Added the ability to select a custom date range.
  •  Fixed `inline_videos` parsing on mobile
  •  Added `source` to desktop & mobile organic results
  •  Fixed `detected_location` in `search_information`
  •  Fixed `date` and `snippet` identification in organic results

⏭️ What's coming next?

  •  Google Maps engine
  •  Amazon Product engine
  •  Integration examples in multiple languages and libraries
  •  LangChain integration (Ruby)

Keep an eye out for further updates! We're dedicated to ensuring you have the best experience. 

During the last month, we added 6 new engines, made significant improvements to the API response time, and made sure it is highly scalable.

🌟 Freshly Released Engines

  ✔️  Google Product Offers​
  ✔️ ​ Google Product Reviews​
  ✔️  Google Product Specifications​
  ✔️  Google Scholar​
  ✔️  Google Scholar Cite​
  ✔️  Google Scholar Profiles​

🔧 Engine Improvements

Google

Introduced an `organic_results` type ​Answer Box field for desktop devices.

Google Play Store

Resolved an issue with the next_page_token. When searching for a specific application, the next_page_token will now reliably return the second page of Similar Products, eliminating previous errors.

Google Trends

Added the ability to select a custom date range.

🚀 Performance Improvements

  • Optimized all key database calls. (Thanks to NewRelic)
  • Refactored API usage tracking. This will allow us to add usage charts.
  • Identified a blocking operation in our API and made it async. Saved `~70ms` on API response time.
  • Performed load testing. We managed to solve all the application and infrastructure issues.
  • Experimented with Rust for HTML parsing. It allowed us to improve the parsing time by up to 10x. This is still very much WIP.

⏭️ What's coming next?

  •  Google Maps engine
  •  Amazon Search engine

  •  YouTube Transcripts engine 

  •  Integration examples in multiple languages and libraries 
  • Langchain integration (Python) 

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

The past month has been another period of high productivity for our team, with considerable upgrades predominantly made to our Google engines. Keep an eye out for more refinements as we persist in adapting and broadening our services in response to your valuable feedback and requirements.

🆕 New Features Released

✔️ Released Google Jobs engine
✔️ Released Google Videos engine
✔️ Released Google Images engine
✔️ Released Google Shopping engine
✔️ Released Google Product engine

⚡ Performance & DevOps

• Improved parsing time up to 4x
• Improved performance for all queries utilizing location geo-targeting. We are able to achieve geo-targetting consistently in 10ms-20ms (took up to 1s previously).
• Integrated CDN with 450+ Points of Presence and 13 regional edge caches in 90+ cities across 49 countries
• Zero-Downtime deployment (Blue-green)
• Integrated Continous Delivery process into our team workflow
• Upgraded infrastructure components to the newest versions available.
• Integrated NewRelic & connected to AWS for enhanced performance monitoring

⏭️ What's coming next?

The main focus is on expanding our selection of available engines and increasing the performance even more. The goal is to have the average response time below 2 seconds consistently. We are happy that a significant part of Google searches are below 1 second already.

• Google Product Offers engine
• Google Product Reviews engine
• Google Product Specifications engine
• Google Maps engine
• Google Scholar engine
• Integration examples in multiple languages and libraries
• More performance improvements

We appreciate you choosing our services. We're working hard to make things better for you. If you need help or have questions, just let us know. We're always here. Thanks for your support.

Our team has had another highly productive month, with significant updates rolling out primarily on our Google Search API. One of the key enhancements is the improved support for mobile and tablet devices across numerous fields. This marks a major leap in our commitment to delivering a seamless and user-friendly search experience on all platforms, making it more convenient and efficient for users to find the information they need. Stay tuned for further improvements as we continue to refine and expand our services based on your feedback and needs.

🆕 New Features Released

✔️ Released Google News engine
✔️ Released Youtube engine

Significant Google Search Engine Update

We're excited to announce that we've significantly expanded our parsing capabilities within the Google Search Engine. The update will incorporate several new fields, mobile and tablet devices support.

New fields in Google Search Engine:

• Jobs
• Local Map
• Inline Tweets
• Inline Recipes
• Inline Images

Mobile and tablet is now supported for given fields:

• Inline Videos
• Jobs
• Top Stories
• Inline Tweets
• Ads
• Local Ads

Additional Updates

In order to further enhance your experience, we have:

• Fixed curl examples
• Changed product_info key in shopping_ads to extensions
• Changed documentation API key description
• In our continuous effort to enhance user experience, we have revised the titles of our API examples

⏭️ What's coming next?

• Google Jobs engine
• Google Maps engine
• Google Videos engine
• Google Images engine
• Google Scholar engine
• Google Shopping engine
• Google Search engine organic results tablet and mobile devices support


Your choice to use our services means a lot to us. We're dedicated to making your experience better. If you need help or have questions, please reach out. We're always here to help. Thanks again for your support.

We have completed another productive month with an emphasis on improving and broadening our Google Search Engine capabilities. Our team has added parsers for more than 100 distinct page blocks, reflecting our ongoing commitment to delivering unparalleled data quality and precision.

🆕 New Features Released

✔️ Released Google Trends engine
✔️ Released Google Trends Autocomplete engine
✔️ Released Google Locations API engine, which enables Google searches with high-precision coordinates across 195 supported countries.

Significant Google Engine Update

We've broadened our parsing scope in the Google engine to include the following fields:

• Ads;

• Local Ads;
• Shopping Ads;
• Inline Videos;
• Inline Shopping;
• Inline Recipes;
• Related Questions;
• Local Results;
• Top Stories;
• Knowledge Graph (with emphasis on Business, Shopping, and Jobs entities).

Additional Updates

In order to further enhance your experience, we have:

• Integrated Intercom live chat support to offer instant assistance.
• Automated embedding of the api_key in all documentation examples when you are signed in for ease of use.
• Resolved issues with product_highlights extraction in Google Play Store.

We thank you for choosing our services and we're committed to continuously improving and providing the best experience for you. If you have any questions or need further assistance, please do not hesitate to reach out.

Though we're a new face in the SaaS world, we're far from inexperienced. For years, we've been providing advanced web scraping solutions with high-performance APIs for our private clients. Now, we're thrilled to make our services available to everyone.

Our mission is simple: to make public data accessible to everyone, with precision, quality, and rich, extensive information at the core of everything we do, while ensuring our APIs perform efficiently and reliably.

Join us on this journey, and as a new startup, we pledge to deliver unparalleled technical support and direct communication with our engineers. This commitment, coupled with our focus on API performance, sets us apart and remains our number one priority and unique value proposition.

🆕 New Features Released

✔️ Google Play Store engine.

✔️ Google Play Product engine.
✔️ Initial version of Google search engine. We parse organic_results and related_queries on the desktop.
✔️ 100 free searches - newly registered users can test our platform with 100 complimentary searches.

⏭️ What's coming next?

• Significant Google search engine improvements. This includes precise search by geolocation and a bunch of new blocks rolling over the next weeks.

• Trends Search API
• Youtube Search API
• Infrastructure upgrades - we're enhancing our infrastructure and conducting stress tests to accommodate even larger scale projects.

Join us today and experience the difference of our advanced web scraping solutions. Welcome aboard!