• 12

A PHP Error was encountered

Severity: Notice

Message: Undefined index: userid

Filename: views/question.php

Line Number: 191


File: /home/prodcxja/public_html/questions/application/views/question.php
Line: 191
Function: _error_handler

File: /home/prodcxja/public_html/questions/application/controllers/Questions.php
Line: 433
Function: view

File: /home/prodcxja/public_html/questions/index.php
Line: 315
Function: require_once

name Punditsdkoslkdosdkoskdo

How to fetch Android Market data when there is no API?

As far as I can tell, there is no API (official or unofficial) to access information about Apps on the Android Market (info such as Title, Icon, Description, Downloads, Comments, etc..) However, there are a few websites that have managed to compile this information anyway.

My question is: How is this being done?

A couple of guesses.

  • Manually (not likely, due to volume)
  • Implemented the protocol used by the official app (by decompiling or observing packets)
  • Running some kind of debugger with the official app to extract the live data
  • Using some kind of automation + screen shots + text recognition (seems complicated)
  • Modifying the (open) source to write a log file

I'm just curious how this was done because I sure can't figure it out. Thanks.

      • 2
    • Just so you know, Java is one of the easiest program to decompile. Decompiling Java program like Android app is a piece of cake. Just Google for it, there is an open source project for that.

There is a Java project on Google code which attempts to build an open source api for the market: http://code.google.com/p/android-market-api/

There're also some JRuby bindings for it: http://github.com/jberkel/supermarket.

However they both require a Google/Market account, and the API is only partially implemented (searching, comments and images at the moment). Also, the reconstructed protocol buffer descriptor (market.proto) could be used to generate bindings for languages other than Java.

  • 32
Reply Report
    • Hmmm... looks promising, I played for a while with Protobuffers but having a ready-made API helps a lot. Once the messages are clear, porting to other platforms should be possible as Protobuffers are multiplatform.
    • I've noticed that using this to grab all data from the market doesn't work. After a number of requests, I got blocked. This really makes me wonder how those websites get their data.

Another option to capture data is using the tcpdump tool that comes with some custom images (like Cyanogenmod and others). You can capture all the packets and then analyze them using Wireshark. This route is easier than hijacking wifi.

Checkout this splendid posts to clear out some things:

Update I have been working on this a bit more and @gsmd is right and they use Protocol Buffers to transfer the information and with the technique described above you can capture the packets. Then you'll need to dig into protobuffers to understand the messages.

  • 7
Reply Report

Unfortunately there is still no official API for Google Play Store (previously known as Android Market) to access Android apps' meta-data (or any other stats).

As mentioned in other answers, you could develop your own web crawler, parse the HTML page and extract the app meta-data you need (e.g. title, descriptions, price, etc). This topic has been covered in this other question.

Another option is to use an open-source library based on ProtoBuf to fetch meta-data about an app, here the link to the project: https://code.google.com/archive/p/android-market-api. This library fetches app meta-data from Google Play on behalf of a valid Google account, but you still need a crawler to "find" which apps are available and schedule their meta-data retrieval. This other open-source project can help you with that: https://code.google.com/archive/p/android-marketplace-crawler.

If you don't want to implement all that by yourself, you could use a third-party service to access Android apps meta-data through a JSON-based API. For instance, 42matters.com (the company I work for) offers an API for both Android and iOS to retrieve apps' meta-data, here more details:


In order to get the Title, Icon, Description, Downloads for an app you can use the "lookup" endpoint as documented here:


This is an example of the JSON response for the "Angry Birds Space Premium" app:

    "package_name": "com.rovio.angrybirdsspace.premium",
    "title": "Angry Birds Space Premium",
    "description": "Play over 300 interstellar levels across 10 planets...",
    "short_desc": "The #1 mobile game of all time blasts off into space!",
    "rating": 4.3046236038208,
    "category": "Arcade",
    "cat_key": "GAME_ARCADE",
    "cat_keys": [
    "price": "$1.15",
    "downloads": "1,000,000 - 5,000,000",
    "version": "2.2.1",
    "content_rating": "Everyone",
    "promo_video": "https://www.youtube.com/embed/g6AL9YqRHaI?ps=play&vq=large&rel=0&autohide=1&showinfo=0&autoplay=1",
    "market_update": "2015-07-03T00:00:00+00:00",
    "screenshots": [
    "created": "2012-03-22T08:24:00+00:00",
    "developer": "Rovio Entertainment Ltd.",
    "number_ratings": 20812,
    "price_currency": "$",
    "icon": "https://lh3.ggpht.com/aQaIEGrmba1ENSEgUtArdm3yhJUug7BRWlu_WaspoJusZyHv1rjlWtYqe_qRjE_Kmh1E=w300",
    "icon_72": "https://lh3.ggpht.com/aQaIEGrmba1ENSEgUtArdm3yhJUug7BRWlu_WaspoJusZyHv1rjlWtYqe_qRjE_Kmh1E=w72",
    "market_url": "https://play.google.com/store/apps/details?id=com.rovio.angrybirdsspace.premium&referrer=utm_source%3D42matters.com%26utm_medium%3Dapi"

Alternatively, we also have a DATA FEED DUMP to access all mobile app data at once: https://42matters.com/docs/app-market-data/file-dump/android-app-details (on this page you can also find a free sample of data for your tests).

I hope this helps, otherwise feel free to get in touch with me. I know this topic quite well and can point you in the right direction.



  • 0
Reply Report