"it seems that the website thinks that i'm a bot: i'm unable to set a proprer user agent" - Because you are a bot, and the probably don't want you scraping their site.