[Bonus] This is a bonus episode. We had too many unreleased episodes in our backlog, so we decided to give you an extra treat this week! Hope you enjoy it!

Join the discussion on how AI agents can intelligently browse websites to retrieve specific information across links and pages. The hosts dive deep into techniques like embedding link text, URL paths, and accessibility attributes to compare against the desired information goal. They explore methods to filter and re-rank links through this embedding approach, accounting for factors like the total number of links per page and whether the target requires navigating across multiple sites.

The conversation covers setting reasonable boundaries on recursive browsing, using a large language model for final relevance assessment, and the broader considerations of efficient agent design for web information retrieval tasks. With their combined 25+ years of AI engineering experience, the hosts provide insights into this emerging and complex challenge.

—
Continue listening to The Prompt Desk Podcast for everything LLM & GPT, Prompt Engineering, Generative AI, and LLM Security.
Check out PromptDesk.ai for an open-source prompt management tool.
Check out Brad’s AI Consultancy at bradleyarsenault.me
Add Justin Macorin and Bradley Arsenault on LinkedIn.

Please fill out our listener survey here to help us create a better podcast: https://docs.google.com/forms/d/e/1FAIpQLSfNjWlWyg8zROYmGX745a56AtagX_7cS16jyhjV2u_ebgc-tw/viewform?usp=sf_link

Hosted on Ausha. See ausha.co/privacy-policy for more information.