Welcome to Brian's Bits, where Brian gets to share at length about various topics stirring inside of him.
 
Remote Control Your Mac With Siri
1 November 2014
 
 
Sections Outline  [+]
How would you like to remote control your Mac via Siri on your iOS device? You could tell iTunes to play a certain playlist, pause or start playback, skip to the next or previous track, raise or lower the volume. Or you could put the screen of your Mac to sleep. Or turn on or off anything connected to a WeMo device. Or more!

Well, I’ve been doing all this for some months. But it’s not like any of this is new or secret — there are a number of other Web sites which give some sort of explanation about how to implement this functionality. However, I have not come across anything that gives comprehensive instructions. Therefore today I will explain to you how you can enjoy this science-fiction-like voice control yourself.

You might think that with the new Dictation Commands feature in OS X Yosemite, you would be able to control iTunes with your voice directly through your Mac without having to use Siri on an iOS device. But according to one report, Dictation Commands and iTunes do not play well together at all. In addition, you would still need to be sitting in front of the computer. Therefore, despite the advancements in OS X, the voice remote control system described in this article still has great value and plenty of usefulness. There are a number of interdependent pieces of the puzzle which all have to be in place before this will work:
 
 
 
Both my Mac and iPad are running on the latest Apple operating systems — OS X 10.10 Yosemite on my Mac and iOS 8.1 on my iPad. But for months I was remote controlling my Mac via Siri under OS X 10.9 Mavericks and iOS 7. However, some of the details for setting all this up may have changed slightly, and the only instructions I am going to give are for Yosemite and iOS 8.1 — mainly because I no longer have machines running the older operating systems to refer to. As I was capturing the two screen shots shown in the next section, I was struck by the realization that I’ve been using my primary e-mail address as my Apple ID for a long time — over 10 years! Now that more of my digital eggs are being gathered into the iCloud basket, maybe that is not such a good idea!

Hopefully I’m not being naïve, but it seems to me that by using my main e-mail address — that has been registered at an incredible number of Web sites over the past 15 years — as my Apple ID, I’m simply making it easier to a hacker to know what my Apple ID is.

It seems reasonable to imagine that by using an e-mail address that I use ONLY for my Apple ID and for no other purpose, I might be adding one thin layer of security that would make my Apple account just a little bit harder to break into.

Therefore, because my Web hosting account with 1 and 1 Internet allows me to set up as many e-mail addresses as I want for any of the domain names I own, it was easy to create a new address just for my Apple ID. But you could just as easily set up a new address with Google or any of the other services which provide free e-mail accounts.

The next step was to sign out of my Apple account in all of the apps and settings through which I had logged in on my computers and iOS devices, which was a bit of a pain. Then I visited Apple’s page for managing my Apple ID, where I was able to change my Apple ID. After that process was completed, I signed back into my Apple account on all my devices using my new Apple ID e-mail address and password.

Even if security experts would tell me that I’m really not any more secure, I feel better about it, and I’m glad I made the change. Although I may not be any more secure, it is possible that it does help. At worst, I’m not any LESS secure than I was before, so I think it was a good decision. Once you have finished reevaluating — and perhaps even changing your Apple ID as I have done — you should verify that you are logged into the same Apple account on both the Mac you want to remote control, and the iOS device you are going to use to perform the remote control. Next, make sure that iCloud synchronization is turned on for the Notes app, both on your Mac and on your iOS device. This is the first piece of the puzzle.
On your Mac, open System Preferences, then open the iCloud settings pane. If you are not logged into iCloud, do so now. Once that is accomplished, make sure there is a check mark next to the Notes app (see the red arrow).
 
 
On your iOS device, open the Settings app, and navigate to the iCloud section. After verifying that you are logged into iCloud, make sure that synchronization for the Notes app is turned on (see the red arrows).
 
The next piece of the voice-remote-control puzzle is using Siri to create a new note on your iOS device. Press and hold the Home Button until Siri activates. Alternatively, if your device is charging, and you have ‘Allow “Hey Siri”’ turned on in Settings, you can simply say “Hey Siri.”

Once you have Siri’s attention, say “Make a note about ___________________.” For our purposes, single words are best. Try filling in the blank with the word “oranges”. Activate Siri again and try “Make a note about bananas.” Try once again with “Make a note about apples.”

Now open the Notes App on your iOS device. You should see the three notes you just created — one with the word “Oranges,” one with “Bananas,” and one with “Apples.”

Next, open the Notes App on your Mac, and if iCloud synchronization is working properly both on your end and on Apple’s end, the three notes should be there as well. If not, you might need to give it a few seconds to synchronize. If after a while the notes are still not on your Mac, you may need to check your settings to make sure the instructions I gave in the previous section have been implemented properly.

Once you have verified that the notes you dictated with Siri on your iOS device are showing up on your Mac, you are ready to implement the next piece of the remote-control puzzle. This is the real brains of the system which allows the previous two steps to actually accomplish something. For over 20 years, Apple has included AppleScript with its operating systems. This powerful programming language is the engine which will power our Siri-Mac remote-control system. First I will give you an overview of what the script we create will accomplish, and then we will look at the specifics of the script itself.

To begin with, let’s give this script a name, so we can more easily refer to it. Following the example of the article which inspired this project, let’s call it “SiriListener.” Once you launch it, SiriListener runs in the background, waiting for a notification from the operating system that a new note has been created.

Once the script receives such a notification, it tells the Notes app to delete that particular note, and then it performs whatever actions we have instructed it to perform for that particular note. Then it goes back to waiting for the next note to be created. Obviously, the Notes app needs to be always running as well.

To make this more concrete, let’s look at a simple example. Let’s say you tell Siri, “Make a note about next.” This creates a new note with the single word “Next” as the contents. Once iCloud has synched that note with your Mac, the SiriListener script is notified. Because the script has been programmed to look for a Note with the contents “Next”, it executes the instructions we have given it for such a case: SiriListener tells iTunes to skip to the next track of whatever music is currently playing. Pretty cool, huh?! The situation in which I usually want to remote control my Mac with Siri is when I’m sitting in my recliner across the room from my computer — reading, relaxing, or whatever. Once things are set up properly as described in this article, it is much easier to use Siri to control iTunes than to use Apple’s iOS Remote app.

Even though I have 800 albums, not all of that music is suitable to play in the background while writing or reading. Furthermore, for much of my background music I have multiple albums from the same artist. Therefore, in order to make my script as simple as possible, I have created playlists in iTunes for the groups of albums I want to remotely play back.

In addition, because of limitations in AppleScript itself, you can’t use a script to tell iTunes to play back a certain album or artist directly. If you want to play a group of tracks, you can only play back a playlist. This is the primary reason why you must put all the music you want to play back via this Siri remote-control system into playlists. This will become clearer once we take a look at the script. In order to see the text of this script better, you should launch the Script Editor app — located in the Utilities folder in the Applications folder. Next, under the Script Editor’s File menu, choose New. Now, back to your Web browser, in the box below, select all of the script text and copy it. Then go back to the new Script Editor window and paste the script text.

Or, rather than copying and pasting the following script, it may be easier for you to simply download the script file: SiriListener.scpt. If you are feeling a bit more adventurous, you might like to download the new, improved, and more-advanced version two: SiriListener2.scpt. Once it is saved on your hard drive, open it in Script Editor and then continue with the following instructions (below the gray box).
-- SiriListener AppleScript app
-- by Brian Byrd, October 2014
-- http://briansbits.com/mac_siri.php

on run
  -- nothing needed here for now...
end run

on idle
  tell application "Notes"

    (********************************************************************************************************************
    ***  SECTION OF COMMANDS TO CONTROL ITUNES PLAYBACK  ****************************************************************
    ********************************************************************************************************************)
    if exists note "Pause" then
      delete note "Pause"
      tell application "iTunes" to pause

    (*******************************************************************************************************************)
    else if exists note "Paws" then
      delete note "Paws"
      tell application "iTunes" to pause

    (*******************************************************************************************************************)
    else if exists note "Stop" then
      delete note "Stop"
      tell application "iTunes" to pause

    (*******************************************************************************************************************)
    else if exists note "Play" then
      delete note "Play"
      tell application "iTunes" to play

    (*******************************************************************************************************************)
    else if exists note "Louder" then
      delete note "Louder"
      tell application "iTunes" to set sound volume to (sound volume + 10)

    (*******************************************************************************************************************)
    else if exists note "Softer" then
      delete note "Softer"
      tell application "iTunes" to set sound volume to (sound volume - 10)

    (*******************************************************************************************************************)
    else if exists note "Back" then
      delete note "Back"
      tell application "iTunes" to back track

    (*******************************************************************************************************************)
    else if exists note "Previous" then
      delete note "Previous"
      tell application "iTunes" to previous track

    (*******************************************************************************************************************)
    else if exists note "Next" then
      delete note "Next"
      tell application "iTunes" to next track

    (********************************************************************************************************************
    ***  SECTION OF COMMANDS TO PLAY PARTICULAR ITUNES PLAYLISTS  *******************************************************
    ********************************************************************************************************************)
    else if exists note "Alex" then
      delete note "Alex"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Alex de Grassi"

    (*******************************************************************************************************************)
    else if exists note "Loreena" then
      delete note "Loreena"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Loreena McKennitt"

    (*******************************************************************************************************************)
    else if exists note "Lorena" then
      delete note "Lorena"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Loreena McKennitt"

    (*******************************************************************************************************************)
    else if exists note "Tony" then
      delete note "Tony"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Tony McManus"

    (*******************************************************************************************************************)
    else if exists note "Sweden" then
      delete note "Sweden"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Vasen"

    (*******************************************************************************************************************)
    else if exists note "Nordic" then
      delete note "Nordic"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Nordic Roots"

    (*******************************************************************************************************************)
    else if exists note "Belt" then
      delete note "Belt"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "John Belt"

    (*******************************************************************************************************************)
    else if exists note "Paganini" then
      delete note "Paganini"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Paganini"

    (*******************************************************************************************************************)
    else if exists note "Ruth" then
      delete note "Ruth"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Ruth Fazal"

    (*******************************************************************************************************************)
    else if exists note "Earth" then
      delete note "Earth"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Back to Earth"

    (*******************************************************************************************************************)
    else if exists note "Parks" then
      delete note "Parks"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "National Parks"

    (*******************************************************************************************************************)
    else if exists note "Native" then
      delete note "Native"
      my TurnOnShuffle()         -- call handler defined below
      tell application "iTunes" to play playlist "Ah Nee Mah"

    (*******************************************************************************************************************)
    else if exists note "I like to move it" then     -- note text NOT case-sensitive
      delete note "I LIKE TO MOVE IT"                -- note text NOT case-sensitive
      -- my TurnOnShuffle()                          -- call handler defined below
      tell application "iTunes" to play playlist "i like to move it" -- playlist name NOT case-sensitive

    (********************************************************************************************************************
    ***  SECTION OF COMMANDS TO CONTROL THE MAC ITSELF  *****************************************************************
    ********************************************************************************************************************)
    else if exists note "Sleep" then
      delete note "Sleep"
      tell me to do shell script "pmset displaysleepnow"

    (********************************************************************************************************************
    ***  SECTION OF COMMANDS TO CONTROL WEMO DEVICES VIA IFTTT  *********************************************************
    ********************************************************************************************************************)
    else if exists note "Office" then
      delete note "Office"
      tell application "Mail"
        set theMessage to make new outgoing message with properties {subject:"#office", visible:false}
        tell theMessage to make new to recipient with properties {address:"trigger@recipe.ifttt.com"}
        send theMessage
      end tell

    (*******************************************************************************************************************)
    else if exists note "Bedroom" then
      delete note "Bedroom"
      tell application "Mail"
        set theMessage to make new outgoing message with properties {subject:"#bedroom", visible:false}
        tell theMessage to make new to recipient with properties {address:"trigger@recipe.ifttt.com"}
        send theMessage
      end tell

    (*******************************************************************************************************************)
    end if     -- end of the If Then / Else If Then / End If block.
    (*******************************************************************************************************************)

  end tell     -- end of the Tell Application "Notes" block.

  return 1     -- part of the On Idle ... End Idle block. Executes the block once every 1 second.

end idle       -- end of the On Idle ... End Idle block.

(***********************************************************************************************************************)
on TurnOnShuffle()                           -- custom AppleScript subroutine to be used above
  activate application "iTunes"              -- need to make iTunes the active window for the simulated mouse click below
  try                                        -- suppress error message if lacking OS X Accessibility permission
    tell application "System Events"         -- a simulated mouse click originates with System Events
      tell process "iTunes"                  -- notice this is process and not application
        click menu item 1 of menu 1 of menu item "Shuffle" of menu 1 of menu bar item "Controls" of menu bar 1
      end tell
    end tell
  end try
end TurnOnShuffle -- end TurnOfShuffle handler
(***********************************************************************************************************************)
 
Now you will want to save this script as an app. Choose File | Export... from the Script Editor menu. In the Save dialog box, choose the location where you want to save this script. I suggest creating a new folder just for scripts. Give this script the name SiriListener.

At the bottom of the dialog box, change the File Format type from Script to Application. This will change the full name of your script to SiriListener.app. You will also want to make sure that the “Stay open after run handler” option is checked. This is critical! SiriListener won’t work if you neglect to check this option! Then click the Save button.

Now we will look at the script in detail. Once you understand how it works, you can modify it for your own purposes. It is beyond the scope of this article to give an in-depth explanation of the AppleScript language. For further reference, see Apple’s AppleScript Language Guide. When programming, it is very useful to be able to make notes in the program and to mark off sections of the program. Almost all programming language allow the programmer to insert comments into the program text, and AppleScript is no exception. According to the AppleScript Language Guide section on comments, there are two types of comments allowed:
  • A block comment begins with the characters  (*  and ends with the characters  *) . These types of comments may be only one line, or they can extend over multiple lines.
  • An end-of-line comment begins with the characters  --  (two hyphens) and ends with the end of the line.
In the SiriListener script, I used block comments with additional asterisks in between to help mark off the various sections of the program. I also use the one-line comments that start with two hyphens in order to make comments about what is happening at certain points in the script. Comments do not cause any actions to be performed — they are merely for the benefit of the reader.

You will notice that the first three lines of this script are comments I wrote, which give information about the script and myself, as well as the Web address of this article. This script uses a basic, easily-extended If Then / Else If Then / End If control structure. Here is a simplified example:
If Note = "oranges" Then
     make orange juice
Else If Note = "apples" Then
     make apple pie
Else If Note = "bananas" Then
     make banana split
End If
Because  if ... if end  is considered a single block, we could think of the initial  if ... then  section, and all the following  else if ... then  sections as sub-blocks within the primary  if ... if end  block. It is important to note that there is no  if end  or  else end  statement at the end of each sub-block. There is only one  if end  statement at the very end of the entire  if ... if end  block.

You can probably see that it is easy to extend this program by adding more options. Immediately above the  if end  statement you could add:
Else If Note = "peaches" Then
     make peach cobbler
And so on and so forth, as much as you like! For the purposes of this script, the order in which you process the various notes does not matter. We could just as well handle the “peaches” note first, and the “oranges” note last. This script also makes use blocks of code called compound statements. Each block begins with a certain keyword, and ends with that same keyword, preceded by the word “end”. Here are some examples:
on run ... end run
on idle ... end idle
tell ... end tell
try ... end try
if ... end if -- (see section above)
Everything you want to happen in that block of code must be placed between the keyword (like “tell” or “try”) and the end keyword statements. If you start a block with a certain keyword, then there MUST be a corresponding end keyword statement, or else the script will not run. But don’t worry, the Script Editor will let you know if the ending statement is missing.

The person who originally wrote the script — which I “borrowed” and transformed into my own — used a lot of  tell "iTunes" ... end tell  blocks just to have iTunes perform a single command each time. I have simplified them into one-line statements which, therefore, don’t need an  end tell  statement at the end of each  tell "iTunes". The original script which I “lifted” started with an  on run ... end run  block. Any statements inside this block will be executed when you first launch the SiriListener app. I have not found anything useful to put there, hence there is only a comment. But I have left this block in the script, in case you come up with something you want to happen at launch time.

At first I thought I should put commands in this block to launch the other apps which this script controls, like Notes, iTunes, and Mail. But I quickly discovered that when the script “tells” an application something, if that app is not running, it is launched automatically. Therefore I removed those unnecessary commands and have left the  on run ... end run  block empty. The  on idle ... end idle  block is the main section of the script. Because this is a Stay-Open application (due to how we saved it) that is always running in the background until you quit it, this idle handler allows the script to receive input from the operating system while it is “hanging around” behind the scenes.

The second-to-the-last line of this block — return 1 — tells the operating system to run the instructions within this  on idle ... end idle  block once every 1 second. One of the most powerful features of AppleScript is its ability to control other Apple programs, parts of the OS X operating system, and even some third-party apps which have provided such control. While in the Script Editor app, click on the top-level menu “Window” and then click on the drop-down menu item “Library”. This will open a window which lists most of the apps and OS X subsystems you can control via AppleScript. If an app does not appear on the list, you can click on the Add button and try adding it.

If you double-click on one of the apps listed in the Library — for example, iTunes — you will open another window which lists all of the commands you can send to iTunes in order to control iTunes through AppleScript. This functionality is the foundation for all of the scripts available at the famous Doug’s AppleScripts for iTunes Web site.

So, with this  tell application “Notes” ... end tell  block of statements, combined with the previous  on idle ... end idle  block, our script is going to check in with the Notes app once each second to see if there are any notes that need to be compared to the list of notes we are looking for. Now we are going to go through the script’s series of  if ... then  and  else if ... then  sub-blocks — one for each different command we want to give through Siri. Because it seemed simpler both for Siri and the AppleScript to handle, I’ve stuck to one-word commands — they are easier to say, and there is less chance Siri will misunderstand. If you want to try multi-word phrases, go for it!

The first sub-block of this  if ... end if  block looks like this:
if exists note "Pause" then
  delete note "Pause"
  tell application "iTunes" to pause
Remember that because this  if ... end if  block is inside the  tell application “Notes” ... end tell  block, we are directing the statements to the Notes app. In the first line, we are asking the Notes app if there is a note whose only contents is the single word “Pause”.

If such a note exists, then we perform the rest of the actions in this  if ... then  sub-block. First, we tell the Notes app to delete this particular note. If we don’t delete it, the next time the script checks in with the Notes app — in one second — it will process this exact same note again, over and over, until the SiriListener app is quit. This is an example of a dreaded infinite loop, which should be avoided at all costs!

Once the note has been deleted, the next line of code — tell application "iTunes" to pause — directs the statement to the iTunes app rather than the Notes app. It also send iTunes just one command — pause — which instructs iTunes to pause whatever media is currently playing. Telling Siri to “make a note about pause” worked fine for a while, until Siri decided one day that I was saying “paws” instead of “pause”. If she only knew how much I dislike pets, she wouldn’t dare confuse the two words! From that day on, every time I said “pause” Siri would make a note about “paws”. Siri is really going to the dogs!

The easiest solution was to add an  else if ... then  sub-block which is activated by the word “Paws” instead of “Pause”. After that slight change to the first two lines, the remaining statement in this new sub-block is identical to the previous  if ... then  sub-block. — tell iTunes to pause.

While I was at it, I decided that perhaps I should avoid the word “pause / paws” altogether. Therefore created a third  else if ... then  sub-block by copying the second one, and then changing the word “Paws” to “Stop”. Now I can have Siri make a note about either “pause / paws” or “stop” to have this script pause the current media playing in iTunes. Looking through the next six  else if ... then  sub-blocks, you can see that if I tell Siri to make a note about
  • “Play” — iTunes will play the currently selected or queued media
  • “Louder” — iTunes will increase the volume by 10 (out of 100)
  • “Softer” — iTunes will decrease the volume by 10 (out of 100)
  • “Back” — iTunes will return to the beginning of the current track
  • “Previous” — iTunes will skip to the previous track
  • “Next” — iTunes will skip to the next track
Next we come to the section of the script in which we tell iTunes to play various playlists. Let’s look at the first  else if ... then  sub-block of this section as an example of how that works.
else if exists note "Alex" then
  delete note "Alex"
  my TurnOnShuffle() -- call handler defined below
  tell application "iTunes" to play playlist "Alex de Grassi"
The first line of the sub-block ask the Notes app if there is a note that consists of the single word “Alex”. If so, the rest of the commands in this sub-block are executed. First, the “Alex” note which brought us to this section of the script is deleted from the Notes app. I’m going to skip the  my TurnOnShuffle()  statement for now and explain it a bit later instead.

The last statement tell iTunes to play a playlist with the name “Alex de Grassi” — into which I had already added my five Alex de Grassi albums. Once these commands are executed by the script, iTunes starts playing the songs in the playlist. All of the subsequent playlist sub-blocks work in the same way.

In the next two  else if ... then  sub-blocks, you will notice that I have two different but similar keywords in order to have iTunes play my Loreena McKennit playlist. That’s because Siri can’t decided how to spell her name. Just like Siri gets confused between “pause” and “paws,” sometimes Siri writes her name as “Loreena” (which is correct), and sometimes as “Lorena” (which is incorrect). In order to cover both cases, I’ve made these two separate entries.

Customizing this script to make it your own is simple! All you need to do is to substitute the one-word keywords like “Alex” and “Loreena” and all the others with your own words or phrases. Then replace my playlist names with the names of your own playlists. I had mentioned above that it is possible to use multi-word phrases with Siri to remote control your Mac instead of using only single words, as I have chosen to do. Just to see how a phrase would work, I added this  else if ... then  sub-block to the script:
else if exists note "I like to move it" then
  delete note "I LIKE TO MOVE IT
  tell application "iTunes" to play playlist "i like to move it"
For the sake of this experiment, I put the song I Like to Move It into a playlist of the same name. Then I told Siri to “make a note about I like to move it”. Within a few seconds, the song was playing — my experiment worked!

Look carefully at the capitalization of “I Like to Move It” each place it occurs in the script. As you can see, I capitalized it differently each time. And because the script performed perfectly, we can concluded that all through the remote-control chain of events — from Siri to Notes to AppleScript to iTunes, capitalization does NOT matter. Or in computer programming lingo, the words used as vocal commands in our system are not case sensitive. This is good because it makes writing the SiriListener script easier.

Of course, when you use a phrase instead of a single word in your Siri note-making, each additional word increases the probability that Siri will misunderstand what you are saying and make a mistake with your note. Obviously you need to use clear, unambiguous phrases which Siri won’t mangle. That’s the main reason I used “I like to move it” for my experiement — I figured that Siri would most likely get it right.

In order to demonstrating this point, I just told Siri to “make a note about Donal Óg”. The note Siri wrote had the text “Donald Ogden”. Ummm ... that’s just NOT going to work!

Because of this, in a couple of instances I purposefully chose certain words which Siri would understand instead of the actual name of an artist, like “Sweden” instead of “Väsen” and “native” instead of “Ah Nee Mah”. As with many things in life, your mileage may vary. When I listen to these playlists, I absolutely want iTunes to play the tracks in random order. Listening to the songs in the same order each times gets really boring really fast! In addition, because my playlists often contain many hours worth of music, I will never hear many of the tracks unless I enable shuffle playback.

In the past you could turn on shuffle via AppleScript. But starting with iTunes 11, this capability no longer exists — a fault which Doug has documented. This shuffle dysfunction continues with iTunes 12.

After researching a fix on the Web, the best (and pretty much only) solution is to use AppleScript’s ability to simulate a click on the iTunes menu item which turns shuffle on. Of course, things are rarely that easy in life. In this case there is a huge snag which takes some effort to get around.
Unfortunately, in the name of “protection” and “security,” Apple has made it difficult to use AppleScript’s ability to simulate a mouse click (and other actions) in other applications.

If you don’t specifically give our SiriListener.app script permission to “control your computer,” trying to turn on iTunes’ shuffle with an AppleScript simulated mouse click will not work. Worse yet, you will get the error message shown to the right.

This is a horrible error message because it doesn’t tell you how to fix the problem. Clicking on the “OK” button merely dismisses the error message. Clicking on the “Edit” button opens the SiriListener script in the Script Editor app. But the problem is not with the script — it’s with an OS X operating system setting.
 
 
More specifically, it all has to do with a check mark in the Accessibility section of the Privacy pane of the Security & Privacy preferences in the System Preferences app. Whew! Let’s unpack that step by step.

On your Mac, launch the System Preferences app. Then double-click on the Security & Privacy icon. Then click on the right-most of the four buttons across the top, labelled “Privacy.” In the list box on the left-hand side of the app, click on “Accessibility.”
 
On the right-hand side of the app is a list of apps (or perhaps an empty list), under the text “Allow the apps below to control your computer.” The image to the right is a screen shot of this settings page on my Mac.

Moom is a very useful little utility that has nothing to do with this article. At first I thought that Script Editor.app needed to be listed here with a check mark, but subsequent testing seems to indicate that it is not necessary. Or course, it can’t hurt to have it here, just in case.

SiriListener.app absolutely needs to have a check mark next to its entry in order for SiriListener to turn on iTunes shuffle. But first you need to click on the lock icon in the bottom-left-hand corner, and then enter your administrative password, in order to make changes on this page.

If these two apps are not in the list, OS X Yosemite has added the plus and minus buttons so you can add to, and delete from, this list yourself. In previous versions of OS X, the operating system itself was in control of which apps are listed here.

If you want, you can add Script Editor.app (in the /Applications/Utilities folder), and SiriListener.app (in whatever folder you decided to save it in). Or you can wait until the first time you use SiriListener to play a playlist. SiriListener’s attempt to turn on iTunes’ shuffle will cause the operating system to add our script app to the list. Then you can simply add a check mark. OK, giving SiriListener.app permission to “control” your computer (in order to turn on shuffle in iTunes via a simulated mouse click on its menu) it not too difficult. You open the “Security & Privacy” settings, click on “Privacy”, click on “Accessibility”, click on the lock icon, enter your password, and then click on the empty checkbox next to SiriListener.app. It’s a bit time consuming, and there are more clicks than I would like, but it’s really not too bad.

Unfortunately, once you turn you Mac off, OS X forgets this setting! Tragic, but true! So every time to start your computer (like I do each morning), you need to go through the process I described in the previous paragraph. OK, it’s not the end of the world, but it is irritating, and frustrating because it should not be necessary. Come on, Apple!

From what I understand, the reason for this “forgetfulness” is that our SiriListener.app is a script and not a “true” app. And not only is it a script, but it is a “homemade” one at that. This means that unless you are a registered Apple developer, SiriListener.app is not code signed.

Because all these details get a lot more technical if we continue to explore them, I’ll stop here. I just wanted to point out that code-signed apps like Moom and Script Editor will retain the check mark you give them in the “Privacy” pane, while our homegrown SiriListener.app won’t. So you have to redo it at every restart of your Mac.

I never merely log out of my Mac user account, so I didn’t know if that had the same effect as restarting the computer. Therefore I just gave it a try, and I’m happy to report that logging out of your user account and then logging back in does not seem to reset the SiriListener.app security permissions. At least that’s some good news! Because the SiriListener script uses the commands to turn on shuffle in iTunes each time an individual playlist is played, rather than repeating those statements each time, I have put them into an AppleScript handler, more commonly known in programming as a subroutine. Here is the script code:
on TurnOnShuffle() --  custom AppleScript subroutine to be used above
  activate application "iTunes"              -- need to make iTunes the active window for the simulated mouse click below
  try                                        -- suppress error message if lacking OS X Accessibility permission
    tell application "System Events"
      tell process "iTunes"
        click menu item 1 of menu 1 of menu item "Shuffle" of menu 1 of menu bar item "Controls" of menu bar 1
      end tell
    end tell
  end try
end TurnOnShuffle -- end TurnOfShuffle handler
 
Without going into too much detail, there are just a few points I want to highlight here. You will notice that most of the statements are within a  try ... try end  block. This has the effect of suppressing any error messages which might result from performing the actions inside the block — like the “Siri Listener is not allowed assistive access” error message I mentioned above.

Of course, if you WANT an error message — to remind you to adjust the security and privacy settings for SiriListener.app — you can remove (or better yet, comment out with two hyphens) the two statements  try  and  try end.

It is interesting to note in the code that it is “System Events” which is applying a simulated click to the menu of “iTunes”, and not iTunes applying the click to itself. Lastly, the long statement which actually performs the simulated click should, as far as I can tell, work with multiple versions of iTunes. Go to your iTunes app, click on the “Controls” top-level menu, then click on the “Shuffle” menu item, and then take a close look at the items in the resulting sub-menu. In the latest version of iTunes (version 12), the first two items are “On” and “Off”. In previous versions (or at least in version 11), the text for these two items was (I’m pretty sure) “Turn On Shuffle” and “Turn Off Shuffle”.

When writing these types of scripts, the different wording of menu items between different versions of an app you are trying to control can cause a previously-working script to not work anymore. Before writing this article, I was using an AppleScript statement to simulate a mouse click on iTunes’ menu which referred to the “Turn On Shuffle” menu item by name. When I upgraded to iTunes 12, I had to modify the script so that it refered to the menu item “On” instead.

As I was writing this article, I researched the possibilities further and found the statement which is now used in the SiriListener script. I like it better because it refers to the menu item to turn on shuffle by its position rather than its name.

Because the position of this menu item is the same in both iTunes 11 and iTunes 12, this statement which turns on shuffle should work in both versions. And as long as Apple doesn’t move the position of this menu item in future versions of iTunes, this statement should continue to work, no matter what text Apple decides to use to label it. Even though remote-voice-control of iTunes is the primary purpose of SiriListener, there are two more sections of the script we have not covered yet: controlling the Mac itself, and controlling a Belkin WeMo device. Because setting up WeMo for our SiriListener involves a lot of details and quite a few steps, I am going to have to save that for a future article. But in closing, I will discuss putting your Mac screen to sleep with a Siri voice command.

Sometimes I go across the room from my Mac to sit in my comfy recliner, only to look back at my Mac and see the dual monitors still shining brightly. Darn! I forgot to put the screens to sleep! But now, with SiriListener, instead of getting up and going across the room to my Mac, all I have to do it hold down the Home button on my iPad Mini, and then tell Siri to “make a note about sleep”. In a second or two the dual monitors go to sleep. Ta-da!
else if exists note "Sleep" then
  delete note "Sleep"
  tell me to do shell script "pmset displaysleepnow"
As you can see, the AppleScript code to perform this sci-fi-like magic is quite simple. Detect a note with the single word “sleep”, delete that note, and then execute a single shell command.

From the little bit of research I have done, it seems that the displaysleepnow argument of the pmset command was introduced only in OS X 10.9 (Mavericks). Therefore, this statement will most likely not work if you are running OS X 10.8 (Mountain Lion) or earlier. Well, that’s one more reason to take advantage of Apple’s free OS X upgrade!

If you remember what we discussed earlier — this is such a long article that it might have been quite a while ago! — because the  if ... end if  block is inside the  tell application “Notes” ... end tell  block, we are directing the statements to the Notes app. Therefore, our Mac screen will not go to sleep if we give the command  do shell script "pmset displaysleepnow"  to the Notes app.

Outside of AppleScript, shell commands are usually given to the Mac through the Terminal app. We could do it that way in our SiriListener script, but there is no need to open the Terminal app just to execute the pmset command. Instead, to direct the statement away from the Notes app, we prefix the statement with the words  tell me to. Simple enough — we’ll just leave it at that. As long as it works, we don’t necessarily need to know all the whys. With a bit of imagination and research, I’m sure that you could invent all sorts of ways to use Siri to remote control your Mac. You could put the entire computer to sleep instead of just the screen, or log off, or restart it, or even shut the computer completely off. You could launch any program, or a group of programs. Or whatever you need.

Well, that’s more than enough for now! I hope you will have fun with this SiriListener script and put it to lots of good uses. I also hope that this article was helpful, informative, and that you came away with a bit useful knowledge about Macs, AppleScript, and iTunes. Be sure to leave your comments and questions in the Feedback section below.

For all of the details about using SiriListener to remote-voice-control WeMo devices, see part two in this series: Using Siri to Control WeMo Devices. If WeMo is not on your radar, you can skip directly to part three: Control Mac Bluetooth and Audio With Siri.

This series ends with part four: AppleScripting the Mac User Interface, in which I explore some of the techniques I used to write the SiriListener script, and some of the programming challenges I had to overcome. I’ve also taken an interesting detour to explain how to set up a virtual aggregate audio output device on a Mac.

Now that we have made science fiction come to life, I’ll leave you with this classic Star Trek movie scene which involves giving voice commands to a Mac. We’ve come a long ways from quaint keyboards, Scotty! “Computer ... ?”
This article is 11th a series of articles on this Web site related to Technology and Computing which also includes (scroll to see the entire list):
1.
26  Oct  2010
2.
11  Jan  2014
3.
24  Jan  2014
4.
29  Jan  2014
5.
5  Feb  2014
6.
7  Feb  2014
7.
14  Feb  2014
8.
15  Feb  2014
9.
16  Feb  2014
10.
17  Feb  2014
11.
Remote Control Your Mac With Siri
1  Nov  2014
12.
12  Nov  2014
13.
20  Nov  2014
14.
22  Nov  2014
15.
2  Dec  2014
16.
6  Dec  2014
Feedback
Your Name:(required — will appear in the comments section below)

Your E-mail Address:(optional — just in case I would like to reply to your comment — will NOT be made public)

Your Web Site:(optional — if entered, a link will appear in the comments section below)
http://
Your Comments:(no HTML, no profanity — will be screened before posting)

Simple Math:(required — demonstrate that you're a human, and not an automated spambot)
What is 1 + 1 ?   
Reader Comments
On February 8, 2015, Gooby wrote:
Hey Dude, I got your Siri Listener app working over the weekend. Neat Stuff. I thought I might be able to play my own playlists by changing the name in the code, but it didn't seem to work...:( Anyway, just writing to say thanks.
On February 8, 2015, Brian wrote:
In response to Gooby’s comments above ... you should be able to play your own playlists with this script. Just make sure that the playlist name in the script matches the playlist name in iTunes exactly. You might want to copy the playlist name in iTunes and paste it directly into your script. Of course, as I mentioned in the article — see Telling iTunes to Play a Playlist — you’ll want to change the keyword that you use in your script to something that matches your playlist.
On August 31, 2015, Lee wrote:
Thank you so much.

Had one issue which was:

tell application "iTunes" to set sound volume to (sound volume 10)

should be:

tell application "iTunes" to set sound volume to (sound volume + 10)

Thank you!!!
On August 31, 2015, Brian wrote:
In response to Lee's comments above ... thanks for the heads up. The missing plus sign is an HTML and database issue. I needed to use the HTML character code for a plus sign instead ... a normal plus sign character disappears in the process.
 
You can also send comments to me privately at: 2bits@briansbits.com
Article Index     |     Search     |     Site Help
< Brian's Photos >< Your Islamic Future >RSS Subscribe