Coordinated crawling via reinforcement learning