Yet another URL -> Markdown service. curl.md's angle is it is focused on developers.
One of the surprising things about converting HTML to Markdown: You often don't need to.
- grab .md files from repos
- append `/index.md` or `.md` to URL
- use 'Accept: text/markdown' header
- GitHub docs article API -> `/api/article?pathname=`
- ... and lots of other shortcuts
You can try out:
- the playground https://curl.md/playground
- do `curl curl.md/example.com` in your terminal
- or open curl.md/example.com in your browser
GitHub if anyone wants to take a look https://github.com/wevm/curl.md
If I understand correctly, my agent just needs to run `curl curl.md/targetsite.com` and it will receive the webpage in an optimized Markdown file format.
Is there any specific depth until which curl.md will crawl?