29th February 2024, 05:03 PM
|
|
Member
|
|
Join Date: Jan 1970
Location: Western Australia
Posts: 3,456
|
|
I fixed mine with help from Discord, i can post what i was told, i had to make changes to the way i was retrieving data, i was previously using Requests with python but i had to change to Selenium so i could create a browser, the browser runs in headless mode, this is what i was told about the issue.
Quote:
by default the user agent is:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36
in headless mode, the user agent is:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/121.0.0.0 Safari/537.36
this contains "HeadlessChrome" which may be blocked, so set the user agent back to the default
options.add_argument("--headless=new")
options.add_argument('--user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"')
|
I know you can use Selenium with excel but i have not done that as yet, here is my python code that gets around this issue.
Quote:
from selenium import webdriver
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument("--headless=new")
options.add_argument('--user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/121.0.0.0 Safari/537.36"')
driver1 = webdriver.Chrome(options=options)
driver1.get("https://api.beta.tab.com.au/v1/tab-info-service/racing/dates/2024-02-26/meetings/R/NOW/races/1/?jurisdiction=VIC")
data = driver1.find_element(By.XPATH, "/html/body").text
print(data)
driver1.close()
|
__________________
One Drive
"If the corporates are treating you poorly , just go elsewhere."
"If they need you , they will soon find out."
"If you need them , you will soon find out." --moeee
_______________________________________________
|