High Performance Web Crawler in Elixir

Fred Wu's Crawler is a web crawler/scraper written in Elixir, focused on high performance. It offers features like worker pooling, rate limiting, and the ability to crawl various asset types. The project also provides extensive configuration options and supports custom modules for more complex use cases. Documentation is available via Hex.

Visit Original Article →