
Hurls par Device1
Hurls short for historic urls is a Firefox Extension that was created so that it'd be easier to interact with the wayback CDX API, and with the purpose of using it in security research.
10 utilisateurs·trices10 utilisateurs·trices
Vous avez besoin de Firefox pour utiliser cette extension
MĂ©tadonnĂ©es de lâextension
Ă propos de cette extension
The user can create a wayback CDX API request with the following:
- Add the domain, by placing
- Add a limit, by placing an integer like "50000", which is the default input, into the limit input, which basically adds "&limit=50000" into the final url. NOTE: The higher this is the more resources are required from your PC!
- Check the subdomain option, that will add "*." before the domain, which will make the CDX API pull results with subdomains aswell.
- Add a date from which point in time the results should be pulled from, by placing a year like "2020" and "2021" into the date inputs, which will basically just add "&from=2020&to=2021" into the final url. According to the CDX API documentation it's also possible to use regular timestamps aswell, however I have not tried that out myself.
- Add various filters available for the API such as, "mimetype" and "statuscode", which allow you to pull more specific type of urls from the CDX API, by pressing the "+ Filter" button and adding the filters such as "mimetype:application/json", which would basically add "&filter=mimetype:application/json" into the final url, allowing you to pull urls that use the corresponding mimetype form the CDX API. There are other filters available that can be read from the CDX API github page: https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server
The results of the request will then be placed into a new tab, which allows you to check the results in various forms:
- You can get all the urls from the response, with no filters or anything placed, by pressing the "All Urls" button.
- You can get all the endpoints from the response, with some regex filters placed to remove trashy endpoints from the results, by pressing the "All Endpoints" button.
- You can get all the parameters from the response, with some regex filters placed to remove trashy parameters from the results, by pressing the "All Parameters" button. This differs from the endpoints in a way as you get all the parameters as single values like "query" rather than "?query=x&anotherparam=y" type of format.
NOTE: The CDX API is very slow and if your target is massive it can take a good while for the request to complete. To somewhat mitigate the wait time you can use a smaller limit or place specific dates to get data from like placing 2023, 2025 into the date input.
- Add the domain, by placing
domain.com
into the domain input, which basically adds "?url=domain.com/*" into the final url.- Add a limit, by placing an integer like "50000", which is the default input, into the limit input, which basically adds "&limit=50000" into the final url. NOTE: The higher this is the more resources are required from your PC!
- Check the subdomain option, that will add "*." before the domain, which will make the CDX API pull results with subdomains aswell.
- Add a date from which point in time the results should be pulled from, by placing a year like "2020" and "2021" into the date inputs, which will basically just add "&from=2020&to=2021" into the final url. According to the CDX API documentation it's also possible to use regular timestamps aswell, however I have not tried that out myself.
- Add various filters available for the API such as, "mimetype" and "statuscode", which allow you to pull more specific type of urls from the CDX API, by pressing the "+ Filter" button and adding the filters such as "mimetype:application/json", which would basically add "&filter=mimetype:application/json" into the final url, allowing you to pull urls that use the corresponding mimetype form the CDX API. There are other filters available that can be read from the CDX API github page: https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server
The results of the request will then be placed into a new tab, which allows you to check the results in various forms:
- You can get all the urls from the response, with no filters or anything placed, by pressing the "All Urls" button.
- You can get all the endpoints from the response, with some regex filters placed to remove trashy endpoints from the results, by pressing the "All Endpoints" button.
- You can get all the parameters from the response, with some regex filters placed to remove trashy parameters from the results, by pressing the "All Parameters" button. This differs from the endpoints in a way as you get all the parameters as single values like "query" rather than "?query=x&anotherparam=y" type of format.
NOTE: The CDX API is very slow and if your target is massive it can take a good while for the request to complete. To somewhat mitigate the wait time you can use a smaller limit or place specific dates to get data from like placing 2023, 2025 into the date input.
Noté 0 par 1 personne
Autorisations et donnéesEn savoir plus
Autorisations nécessaires :
- Accéder aux onglets du navigateur
- Accéder à vos données pour tous les sites web
Plus dâinformations
- Version
- 1.0.1
- Taille
- 11,84Â Ko
- DerniĂšre mise Ă jour
- il y a 3 mois (6 mai 2025)
- Catégories associées
- Licence
- Licence MIT
- Historique des versions
- Ajouter Ă la collection
Notes de version pour la version 1.0.1
- Added default dark mode
- Tiny UI changes
Plus de modules créés par Device1
- Il nây a aucune note pour lâinstant
- Il nây a aucune note pour lâinstant
- Il nây a aucune note pour lâinstant
- Il nây a aucune note pour lâinstant
- Il nây a aucune note pour lâinstant
- Il nây a aucune note pour lâinstant