What's New

New updates and improvements to SearchApi

April 2024: Update

New
This month, we released a major addition to our Google Maps API suite: the Google Maps Place API. Alongside this, we added four additional engines, a new search analytics feature, and several engine improvements.

🌟 Freshly Released Engines:


Our new Google Maps Place API lets you access detailed information about specific locations, including names, addresses, ratings, reviews, admissions, nearby popular places, and more!

With the Photos API, it is easy to integrate high-quality images of places into your applications. This API provides access to millions of photos, helping you create more visually engaging user experiences.


This API offers insights into the top trending topics and queries on Google Trends, updated daily. It's perfect for keeping your content relevant and engaging.

Get real-time access to the hottest trends with Google Trends Trending Realtime API. This API provides instant updates on the latest trending topics and queries, allowing you to react quickly to emerging trends and capture your audience's attention.

Access real-time price changes, historical data, quarterly and annual company financials, exchange rates for major world currencies, top news, and key events impacting price movement.

🔎 Search Analytics Page (Scale Plan and above)



We've launched a new search analytics feature that provides insights into the performance of search engines over time. It allows tracking and analyzing key metrics such as average response speed, total number of searches, success rate, and error rate. The data is visualized through interactive charts, displaying daily search volumes and breaking down performance for average speed and success rate every hour. This feature is available for Scale plans and higher.

🔧 Engine Improvements:


Google Search API:

  • Added a new parsing field - Questions and Answers.
  • Added a new parsing field - Events.
  • Fixed an issue where inline_images were not parsed correctly.

Bing Search API:

  • Fixed a Bing API request issue that caused timeouts.

Google Maps API:

  • Removed the ll request parameter from the list of required parameters.

Amazon Product API:

  • Fixed an issue where the specifications_list was not parsed correctly.

Google Finance API:

  • Fixed an issue where the graph results displayed default results for 1 month (1M) instead of 1 day (1D).

Google Images API:

  • Now returns full image titles instead of shortened ones.

Other improvements:

  • Added enterprise plans.
  • Sped up the Search History page by adding missing database indexes.
  • Added "Time Taken" column in Search History.

⏭️ What's coming next?


  • Google Maps Reviews API
  • YouTube Reviews API
  • YouTube Channel API
  • YouTube Trending API
  • Playground
  • Organisational features

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

March 2024: Update

New
Today, we're excited to share the latest improvements and give you a sneak peek into what’s coming next.

🌟 Freshly Released Engines:





Google Maps allows searching for businesses using latitude and longitude GPS coordinates. Collect addresses, ratings, phone numbers, and websites. Improve local SEO by tracking business listing positions, and gathering ad information.

The Google Patents API enables retrieval of patent information, including titles, abstracts, claims, and descriptions. This API is crucial for research, product development, and competitive analysis.

The Google Patents Details API offers detailed information on individual patents, such as publication and application dates, inventors, and assignees.

The YouTube Video API allows for searching videos, retrieving video details, and accessing titles, descriptions, and related videos. 

🔄 Early Renewal




We've introduced a new feature that automatically renews your plan if all plan searches are used. This ensures uninterrupted access to our APIs and services so you can continue your work without any issues. No need to wait until the end of your billing cycle – we've got you covered.

đź”— LangFlow Integration:


​LangFlow​ is a dynamic graph where each node is an executable unit. Its modular and interactive design fosters rapid experimentation and prototyping, pushing hard on the limits of creativity. We have added a custom component allowing you to call any engine we support. We are looking forward to adding more examples and tutorials about it in the future. 

🔧 Engine Improvements:


Google Trends API:

  • Completely refactored the engine. JSON keys were left unchanged. You can now enjoy stable and fast response times without any errors.
  • Fixed `now 7-d` parameter support.

Google Search API:

  • Improved parsing speed on mobile devices. There was a bug causing significantly higher response times compared to desktop.
  • Fixed ​About this result field.

Google News API:

  • Fixed top stories extraction.

Bing API:

  • Fixed an issue causing some requests to return an error status.

YouTube API:

  • Fixed an issue causing several requests to return error status.

YouTube Transcripts API:


Other improvements:

  • Fixed documentation anchor links. Parameters can now be selected as anchor links as well.
  • Improved sidebar navigation in documentation pages.

⏭️ What's coming next?


  • Google Maps Places API
  • Google Maps Reviews API
  • Google Maps Photos API
  • Google Finance API

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

February 2024: Update

New
This month, we’ve rolled out several updates that we think you’ll find valuable: a new integration, significant engine and performance improvements, and a fresh search history feature.

🌟 ​Bing Search Engine​


We're excited to release Bing Search API offering organic results, ads, local results, related questions, and related searches. Similarly to Google, it supports precise location-based targeting using our ​Locations API. If that's not enough, you can also use latitude and longitude coordinates for coordinate-level precision. 

To ease integration, JSON keys align closely with those of the Google Search API. Perfect for building tools across different search engines.

🚀 Major Performance Upgrade


We've optimized our API, focusing on speed and stability. Key steps included revamping our proxy network on January 4th and boosting server capacity while fine-tuning our proxy network on a per-engine basis on January 12th, thanks to December’s backend efforts on the admin side.

As a result, SearchApi average response time is already less than 3 seconds. We anticipate improving it further to bring it below the 2-second threshold.

🔎 Search History Page


A new feature to review recent searches with the ability to filter by the search engine. We hope it will be really useful in debugging automated queries.

🔧 Engine Improvements:


Google Search API:

  • Added a new major parsing field: Sports Results. This includes three different types of sports results: 
    • Athlete Stats: This type can be triggered when searching for player statistics.
    • Team Sports: This type can be triggered when looking for games in basketball, football, or other team sports.
    • Single Sports: This type can be triggered when searching for golf, F1, tennis, or other individual sports games.



Google Shopping API:

  • Fixed product link and added `prds` value when there is no `product_id`.
  • Fixed documentation num parameter, as Google Shopping does not support the num=10 parameter and returns 60 results instead.

Google Product API:

  • Fixed an issue where the selected Product didn't return the main image of it.

Google Scholar API:

  • Fixed issues setting `num=20`.

Google Lens API:

  • Added mobile support as Google Lens returns larger images on it.
  • Added exact price and currency to visual_matches.
  • Added price, and stock information to exact_matches. Fixed source.
  • Fixed HTML. Now HTML shows an image, which was searched for as well.
  • Added ​country​ parameter. It allows making searches in a specific country to influence shopping results and the accuracy of place identification. 

Google News API:

  • Fixed top stories extraction.

Google Trends API:

  • Google Trends API is fully functional again.

Youtube Transcripts API:

  • API now returns all available languages even if the transcripts are not available for selected languages. Check ​documentation for an API example.

Amazon Product API:

  • Added bundle items & bundle carousel.
  • Improved variants extraction.
  • Improved buybox extraction.
  • Improved compare_with_similar table parsing.

đź”— New Integrations:


  • ​search_with_lepton​ - A dynamic, LLM-supported conversational search tool with an intuitive interface and shareable, cached results for efficient information retrieval.

Other improvements:


  • Transitioned to Stripe Embedded Checkout.
  • Upgraded to Ruby 3.3 in our production environment.

⏭️ What are we working on next?


  • Google Maps API
  • YouTube Channel API
  • Amazon Reviews API
  • Consider a CORS policy update for easier API experimentation.

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

January 2024: Update

New
Happy New Year! We're excited to start 2024 by improving your experience with SearchApi. This month, we've worked hard to improve our engines, especially Google. We believe these improvements will make your work easier and help you discover new things more efficiently.

🔧 Engine Improvements:


Google:

  • Added "Discussions and Forums". The element shows forums and their topic based on your query.
  • Implemented a new case of sitelinks - list - for organic results.
  • Added new element "From sources across the web" that shows sites that cover a specific topic in this grid list view.
  • Fixed local_ads and ads on mobile and desktop.
  • Fixed inline_images.
  • Fixed top_stories.
  • Added weather_results on mobile for up-to-date weather information.
  • Fixed the snippet in organic_results. Google started testing longer snippets, which might be related. 

Google News:

  • Fixed top_stories.

Google Shopping:

  • Fixed an issue when filters didn't return any text.
  • Added Google Shopping pagination.

Google Product:

  • Fixed an issue where typical_prices didn't return any results.

Google Lens:

  • New fields are being parsed - videos, related_searches, and exact_matches.

👨🏻‍💻 Product Changes



⏭️ What are we working on next?


  • Testing Bing API & Google Maps API that will be released this month.
  • Work in progress for API usage stats and search history features, initially for the admin side, with plans to extend to user accounts.
  • Even better API response time. We are in the process of optimizing our proxy pool. Thanks to the work done on the admin side. 
  • More integrations. 

Stay tuned for more updates! As always, we're committed to providing you with the best experience. Got feedback or questions? Just hit reply.

December 2023: Update

New
We're excited to bring you our latest updates for this month, including two new integrations, a major engine update, and an affiliate program.

🌟 New Engines


This is a single reason our update is late this month. We wanted to polish and release it before Christmas. 🎄 

Use the Amazon Product API to scrape product details, prices, delivery information, ratings, reviews, and more. It supports 22 Amazon domains and retrieves product information by ASIN.

We made sure to parse as much information as possible on the page and we think this is one the most comprehensive parsers on the market right now. We will still polish it during the week, so expect some minor changes. 

Customizing the delivery location country and zipcode is one of the key features we will be exploring at the start of 2024 and expanding the Amazon support further. 


Use the Google Autocomplete API to scrape real-time search autocomplete suggestions, their relevancy, popularity, and type. It supports nine different clients, including Chrome, Firefox, and Safari, each offering unique autocomplete results. Google Autocomplete API comes with a customizable cursor position, allowing different variations of suggestions.

đź’° Affiliate Program (30% revenue share)


Promote SearchApi and earn a 30% revenue share for every new customer you refer.

We partnered with Rewardful as our affiliate software. We thoroughly tested our system, so it correctly tracks all people registering on the platform, even through the Google and Github logins. Learn more about the program on the ​affiliate program page​.

After signing up you get immediate access to your personal affiliate link and promotion materials, which you can use to recommend the SearchApi. Building integrations, self-hosted tools, and writing or organizing webinars are just a few ways to get started. Head over to the ​affiliate page​ for more ideas.

🔧 Engine Improvements:


Google

  • The new "Things to Do" block displays the most popular places to visit in the selected location.




Google Scholar

  • Improved snippets to return full text hidden under "show more" in DOI or full article title searches.




đź”— Integrations


  • ​​SerpBear​ - a simple, self-hostable, open-source SERP position tracking app. It allows you to track your website's keyword positions in Google and get notified of their positions.
  • ​Langchain​ (Python) - when integrating SearchApi into the Langflow, we noticed some issues, mainly with imports. This ​PR​ fixed the issues and has been released in ​the v0.0.346​ version.
  • ​HayStack​ - LLM orchestration framework to build customizable, production-ready LLM applications. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search, or conversational agent chatbots.

Other changes

  • Improved accessibility on the landing and documentation pages.
  • Improved loading speed for the documentation pages.
  • SEO improvements: markup & semantic tags.


⏭️ What are we working on next?

  • Development of Bing Search API and Google Maps API.
  • A status page to display our uptime and performance metrics.
  • Work in progress for API usage stats and search history features, initially for the admin side, with plans to extend to user accounts.
  • More integrations are underway. 

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

November 2023 Update

New
As another month wraps up, we're eager to share our latest strides in enhancing your development workflow.

🛠️ Developer Experience


  • Integration Examples. Expanded support for 13 programming languages, enhancing our library support to facilitate seamless integration with SearchApi. We've introduced intuitive icons for quick actions, enabling you to copy code snippets or open API examples in a new tab more efficiently.
​​



  • API Key Management
    • Multiple API Keys. ​The new Account API keys page​ allows the creation of distinct API keys for different environments or individual team members, reinforcing your operational security posture.
    • Zero downtime rotation. Transition between old and new API keys without service interruption, ensuring continuous operation.
    • Default API key. The default API key is used sitewide across all integration examples, so you can test the API faster.
    • The default API key is now hidden in the dashboard. 

🔧 Engine Improvements


Google
  • New "Salary Estimates" block providing valuable compensation data.



  • New "About this result" block offering contextual information about search results.



  • "People also search for" block is now parsed as `related_searches` on mobile.
  • Added pagination for Google searches, News, and Jobs.

Google Lens
  • Now supports query parameters in image URLs, correcting previously encountered encoding issues.

👨‍💻 DevOps


  • Resolved 504 HTTP response timeouts by adjusting the CloudFront to accommodate longer wait times for AWS ALB responses and established CloudWatch alerts for proactive monitoring.
  • Upgraded our infrastructure to utilize Rails 7.1.1 & Ruby 3.2.2 for improved performance and stability.

đź”— New Integrations



⏭️ What's coming next?


  • Amazon Product API
  • Google Maps API
  • TikTok API
  • More integrations

Your engagement is crucial for our continuous improvement. We encourage you to test these updates and look forward to your valuable feedback. Thank you for being an integral part of the SearchApi developer community.

October 2023 Update

New
We're thrilled to update you on some of the newest enhancements and give you a sneak peek into what’s coming next.

🦜🔗 ​LangChain​ Integration

Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. However, using these LLMs in isolation is often insufficient for creating a truly powerful app - the real power comes when you can combine them with other sources of computation or knowledge.

By embedding the SearchAPI, the Large Language Model can respond by querying real-time data to answer questions about current events. Its usage is particularly interesting with the "Self-ask with search" agent. This agent is equivalent to the original ​self-ask with search paper​, where a Google search API was provided as the tool.

Another common user case is summarizing long documents that naturally run into context window limitations. Here is an example ​using SearchApi to load a lengthy YouTube video transcript​, then summarize and distill it with a refine chain.

SearchApi is now natively available as part of ​langchainjs​ and ​langchain​ libraries.

For a quick overview, check out this tweet on SearchApi integration in LangChainJS.

Here is the official SearchApi documentation on its usage in Python and Javascript:
  •  Python examples​
  •  JavaScript/TypeScript examples​​

🌟 Freshly Released Engines:

  ✔️  Google Lens API​ - shipped in less than 24 hours based on ​@levelsio tweet on Twitter​. Useful as a tool in ​affiliate marketing​ for fetching product sources, links, prices, and stock information.
  ✔️  YouTube Transcripts API​
  ✔️  Amazon Search API​

🔧 Engine Improvements:

Google
  •  Added the ability to select a custom date range.
  •  Fixed `inline_videos` parsing on mobile
  •  Added `source` to desktop & mobile organic results
  •  Fixed `detected_location` in `search_information`
  •  Fixed `date` and `snippet` identification in organic results

⏭️ What's coming next?

  •  Google Maps engine
  •  Amazon Product engine
  •  Integration examples in multiple languages and libraries
  •  LangChain integration (Ruby)

Keep an eye out for further updates! We're dedicated to ensuring you have the best experience. 

September 2023 update

New
During the last month, we added 6 new engines, made significant improvements to the API response time, and made sure it is highly scalable.

🌟 Freshly Released Engines

  ✔️  Google Product Offers​
  ✔️ ​ Google Product Reviews​
  ✔️  Google Product Specifications​
  ✔️  Google Scholar​
  ✔️  Google Scholar Cite​
  ✔️  Google Scholar Profiles​

🔧 Engine Improvements

Google
Introduced an `organic_results` type ​Answer Box field for desktop devices.

Google Play Store
Resolved an issue with the next_page_token. When searching for a specific application, the next_page_token will now reliably return the second page of Similar Products, eliminating previous errors.

Google Trends
Added the ability to select a custom date range.

🚀 Performance Improvements

  • Optimized all key database calls. (Thanks to NewRelic)
  • Refactored API usage tracking. This will allow us to add usage charts.
  • Identified a blocking operation in our API and made it async. Saved `~70ms` on API response time.
  • Performed load testing. We managed to solve all the application and infrastructure issues.
  • Experimented with Rust for HTML parsing. It allowed us to improve the parsing time by up to 10x. This is still very much WIP.

⏭️ What's coming next?

  •  Google Maps engine
  •  Amazon Search engine
  •  YouTube Transcripts engine 
  •  Integration examples in multiple languages and libraries 
  • Langchain integration (Python) 

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

August 2023 Update

New
The past month has been another period of high productivity for our team, with considerable upgrades predominantly made to our Google engines. Keep an eye out for more refinements as we persist in adapting and broadening our services in response to your valuable feedback and requirements.

🆕 New Features Released

✔️ Released Google Jobs engine
✔️ Released Google Videos engine
✔️ Released Google Images engine
✔️ Released Google Shopping engine
✔️ Released Google Product engine

⚡ Performance & DevOps

• Improved parsing time up to 4x
• Improved performance for all queries utilizing location geo-targeting. We are able to achieve geo-targetting consistently in 10ms-20ms (took up to 1s previously).
• Integrated CDN with 450+ Points of Presence and 13 regional edge caches in 90+ cities across 49 countries
• Zero-Downtime deployment (Blue-green)
• Integrated Continous Delivery process into our team workflow
• Upgraded infrastructure components to the newest versions available.
• Integrated NewRelic & connected to AWS for enhanced performance monitoring

⏭️ What's coming next?

The main focus is on expanding our selection of available engines and increasing the performance even more. The goal is to have the average response time below 2 seconds consistently. We are happy that a significant part of Google searches are below 1 second already.

• Google Product Offers engine
• Google Product Reviews engine
• Google Product Specifications engine
• Google Maps engine
• Google Scholar engine
• Integration examples in multiple languages and libraries
• More performance improvements

We appreciate you choosing our services. We're working hard to make things better for you. If you need help or have questions, just let us know. We're always here. Thanks for your support.

July 2023 Update

New
Our team has had another highly productive month, with significant updates rolling out primarily on our Google Search API. One of the key enhancements is the improved support for mobile and tablet devices across numerous fields. This marks a major leap in our commitment to delivering a seamless and user-friendly search experience on all platforms, making it more convenient and efficient for users to find the information they need. Stay tuned for further improvements as we continue to refine and expand our services based on your feedback and needs.

🆕 New Features Released

✔️ Released Google News engine
✔️ Released Youtube engine

Significant Google Search Engine Update

We're excited to announce that we've significantly expanded our parsing capabilities within the Google Search Engine. The update will incorporate several new fields, mobile and tablet devices support.

New fields in Google Search Engine:

• Jobs
• Local Map
• Inline Tweets
• Inline Recipes
• Inline Images

Mobile and tablet is now supported for given fields:

• Inline Videos
• Jobs
• Top Stories
• Inline Tweets
• Ads
• Local Ads

Additional Updates

In order to further enhance your experience, we have:

• Fixed curl examples
• Changed product_info key in shopping_ads to extensions
• Changed documentation API key description
• In our continuous effort to enhance user experience, we have revised the titles of our API examples

⏭️ What's coming next?

• Google Jobs engine
• Google Maps engine
• Google Videos engine
• Google Images engine
• Google Scholar engine
• Google Shopping engine
• Google Search engine organic results tablet and mobile devices support


Your choice to use our services means a lot to us. We're dedicated to making your experience better. If you need help or have questions, please reach out. We're always here to help. Thanks again for your support.



June 2023 Update

New
We have completed another productive month with an emphasis on improving and broadening our Google Search Engine capabilities. Our team has added parsers for more than 100 distinct page blocks, reflecting our ongoing commitment to delivering unparalleled data quality and precision.

🆕 New Features Released

✔️ Released Google Trends engine
✔️ Released Google Trends Autocomplete engine
✔️ Released Google Locations API engine, which enables Google searches with high-precision coordinates across 195 supported countries.

Significant Google Engine Update

We've broadened our parsing scope in the Google engine to include the following fields:

• Ads;
• Local Ads;
• Shopping Ads;
• Inline Videos;
• Inline Shopping;
• Inline Recipes;
• Related Questions;
• Local Results;
• Top Stories;
• Knowledge Graph (with emphasis on Business, Shopping, and Jobs entities).

Additional Updates

In order to further enhance your experience, we have:

• Integrated Intercom live chat support to offer instant assistance.
• Automated embedding of the api_key in all documentation examples when you are signed in for ease of use.
• Resolved issues with product_highlights extraction in Google Play Store.

We thank you for choosing our services and we're committed to continuously improving and providing the best experience for you. If you have any questions or need further assistance, please do not hesitate to reach out.

May 2023 Update

New
🚀 Official Launch Announcement

Though we're a new face in the SaaS world, we're far from inexperienced. For years, we've been providing advanced web scraping solutions with high-performance APIs for our private clients. Now, we're thrilled to make our services available to everyone.

Our mission is simple: to make public data accessible to everyone, with precision, quality, and rich, extensive information at the core of everything we do, while ensuring our APIs perform efficiently and reliably.

Join us on this journey, and as a new startup, we pledge to deliver unparalleled technical support and direct communication with our engineers. This commitment, coupled with our focus on API performance, sets us apart and remains our number one priority and unique value proposition.

🆕 New Features Released

✔️ Google Play Store engine.
✔️ Google Play Product engine.
✔️ Initial version of Google search engine. We parse organic_results and related_queries on the desktop.
✔️ 100 free searches - newly registered users can test our platform with 100 complimentary searches.

⏭️ What's coming next?

• Significant Google search engine improvements. This includes precise search by geolocation and a bunch of new blocks rolling over the next weeks.
• Trends Search API
• Youtube Search API
• Infrastructure upgrades - we're enhancing our infrastructure and conducting stress tests to accommodate even larger scale projects.

Join us today and experience the difference of our advanced web scraping solutions. Welcome aboard!