{"id":77549,"date":"2024-10-30T03:29:30","date_gmt":"2024-10-30T07:29:30","guid":{"rendered":"https:\/\/blog.cyberconservices.com\/?p=77549"},"modified":"2024-10-27T14:37:16","modified_gmt":"2024-10-27T18:37:16","slug":"how-anthropics-new-computer-use-ability-could-further-ai-automation","status":"publish","type":"post","link":"https:\/\/blog.cyberconservices.com\/index.php\/2024\/10\/30\/how-anthropics-new-computer-use-ability-could-further-ai-automation\/","title":{"rendered":"How Anthropic’s new ‘computer use’ ability could further AI automation"},"content":{"rendered":"

By Anirban Ghoshal <\/a>\u00a0– Anthropic\u2019s Claude 3.5 Sonnet large language model has gained a new ability: operating a computer.<\/p>\n

The new ability, which the company is calling \u201ccomputer use,\u201d is currently in beta test. It enables developers to instruct Claude 3.5 Sonnet, through the Anthropic API<\/a>, to read and interpret what\u2019s on the display, type text, move the cursor, click buttons, and switch between windows or applications \u2014 much as today\u2019s\u00a0robotic process automation<\/a>\u00a0(RPA) tools can be instructed \u2014 much more laboriously \u2014 to do.<\/p>\n

\n
\n
\n

To apply its ability to use a computer, Claude 3.5 Sonnet starts from a prompt defining its goal, identifies the steps necessary to reach that goal, and then scans screenshots much as a human would look at the screen of a computer to figure out how to perform those steps.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

\n
\n
\n

Key to that is Claude 3.5 Sonnet\u2019s new-found ability to return the coordinates of a feature in an image, enabling it to position the cursor on a button or in a text box on screen.<\/p>\n

Claude 3.5 Sonnet needs definitions of the tools and software on the computer it will operate, and authorization to access them. It then sends requests to use the tools, and examines the response to see if has succeeded or whether it needs to continue using the tool to complete its task.\u00a0 Read On:<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Anthropic\u2019s has upgraded its Claude 3.5 Sonnet LLM with a new ability, computer use, opening up new opportunities in robotic process automation (RPA) and more.<\/p>\n Continue reading →<\/span><\/a>","protected":false},"author":1,"featured_media":77552,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[674,628],"tags":[675,629],"class_list":["post-77549","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-robotic-process-automation-processs-automation","category-rpa","tag-robotic-process-automation","tag-rpa"],"_links":{"self":[{"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/posts\/77549","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/comments?post=77549"}],"version-history":[{"count":1,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/posts\/77549\/revisions"}],"predecessor-version":[{"id":77553,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/posts\/77549\/revisions\/77553"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/media\/77552"}],"wp:attachment":[{"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/media?parent=77549"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/categories?post=77549"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.cyberconservices.com\/index.php\/wp-json\/wp\/v2\/tags?post=77549"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}