18

I want to scrape the data from an ArcGIS map. The following map has a popup when we click the red features. How do I access that data programmatically?

Link : https://cslt.maps.arcgis.com/apps/MapSeries/index.html?appid=2c9f3e737cbf4f6faf2eb956fa26cdc5

0

1 Answer 1

34

Note: Please respect the access and use constraints of any ArcGIS Online item you access. When in doubt, don't save a copy of someone else's data.

The ArcGIS Online REST interface makes it relatively simple to get the data behind ArcGIS Online items. You need to use an environment that can make HTTP requests and parse JSON text. Most current programming languages either have these capabilities built in or have libraries available with these capabilities.

Here's a general workflow that your code could follow.

  1. Use the app ID and the item data endpoint to see the app's JSON text:

    https://www.arcgis.com/sharing/rest/content/items/2c9f3e737cbf4f6faf2eb956fa26cdc5/data

  2. Search that text for webmap and see that the app uses the following web maps:

    • d2b4a98c39fd4587b99ac0878c420125
    • 7b1af1752c3a430184fbf7a530b5ec65
    • c6e9d07e4c2749e4bfe23999778a3153
  3. Look at the item data endpoint for any of those web maps:

    https://www.arcgis.com/sharing/rest/content/items/d2b4a98c39fd4587b99ac0878c420125/data

  4. The list of operationalLayers specifies the feature layer URLs from which you could harvest data. For example:

    https://services2.arcgis.com/gWRYLIS16mKUskSO/arcgis/rest/services/VHR_Areas/FeatureServer/0

  5. Then just run a query with a where of 0=0 (or whatever you want) and an outFields of *:

    https://services2.arcgis.com/gWRYLIS16mKUskSO/arcgis/rest/services/VHR_Areas/FeatureServer/0/query?where=0%3D0&outFields=%2A&f=json

    Use f=html instead if you want to see a human-readable request form and results.

    Note that feature services have a limit of how many features you can get per request, so you will probably want to filter by geometry or attribute values. Read the documentation to learn everything you can do with feature service queries.

8
  • 1
    I'm glad my answer helped. May 8, 2018 at 8:22
  • 1
    Is this answer still valid? I was looking for data sources here. Found no webmap, just mapWidget, then datasets.dataSource.id – which looks like this: 1200f3f1-8f72-4ea6-af16-14f19e9a4517 – it doesn't seem to work with /sharing/rest/content/items/<itemId>/data requests
    – pax
    Mar 14, 2020 at 19:28
  • 5
    Yes, this answer is valid. The item you're looking at is a dashboard, which is a different structure than the story map that the OP asked about. In the dashboard item data, there are three mapWidget objects, each with an itemId that is a web map item ID. You can use any of those items' data (here, for example; the other two seem almost identical to me) and start at step 3 in this answer. Mar 15, 2020 at 20:36
  • 1
    This was incredibly helpful!
    – jlev
    Jun 10, 2020 at 21:44
  • 1
    I'm having some problems to retrieve data from a dashboard but this is REALLY helpful
    – Nacho R
    Feb 18, 2022 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.