Why Every Creative Needs To Know a Little Unix

The Mac OS has developed into a truly powerful and stable operating system. Long gone are the day of System 7 Type 11 errors that would bring your machine (and your productivity) to a grinding halt. And yet, all that power has not come at the expense of usability. Apple have done an outstanding job of shielding the end user from all of the UNIX, Apache and other hard-core technologies Mac OSX is built upon. Apple have always been known for wrapping their products in an attractive Graphical User Interface (GUI) thus allowing even the most digitally retarded end user to jump in without feeling overwhelmed.

Creative types long ago adopted the Mac as their platform of choice because it allowed them to create without having to learn any type of scripting, command-prompting or back-door hackery. We gladly traded off those ‘Ultimate Power User’ features that fans of Windows and Linux would gloat over. No, Mac users have been content to just turn on the box and get to work, joyfully ignorant of such shenanigans.

However, there are times when the raw power sitting beneath the sheen of a user-friendly GUI needs to be accessed directly. Sometimes the shiny user-friendly GUI gets in the way just a little too much. Fortunately, Apple have provided several windows (no pun intended) into the machine room of Mac OSX, where one can tinker with the cogs and pulleys that make The Big Cat roar. The easiest and most obvious way into the land of UNIX is via the Terminal, found in every Mac OSX Utilities folder (Cmd-Shift-U from the Finder). The Terminal is an application that lets the user open a ‘shell‘ to run UNIX command line instructions and gain greater access than a normal admin account can.

I’ve dabbled in UNIX commands in the Terminal a little. Listening to the Mac Geek Gab piqued my interest and I taught myself a few minor tricks. Nothing too fancy. This week I found myself depending on Terminal to save my bacon, because the ‘nuke and pave’ I performed last week left my MacPro in a state of Permissions meltdown. Actually, it was the Migration Assistant pulling my User data back that made a mess of things. Luckily I found a few really useful tools to force my belligerent machine to behave again.

The Due Diligence Backstory

My system had been acting strangely; spinning beach balls, very slow processes, until it locked up on me. After a forced shut-down and reboot I found three of my four hard drives were inaccessible. These were my video Capture Scratch drive, my Clients and Projects drive and my Time Machine back-up drive. That’s 5TB of data I was locked out of. Needless to say I was a tad nervous.

locked-drive.png

I spent a frustrating two days running a concoction of Onyx, Cocktail, DiskUtility and DiskWarrior in an effort to wrestle the permissions into shape. Alas, nothing I tried returned access to my data. In the Get Info windows I kept trying to reset ownership, but something in the ACL (Access Control List) was corrupt and I could not fix the problem. In the Get Info window I noticed the Hard Drive icon was showing a padlock, even though the ‘Locked’ option was unavailable.

I knew from listening to the Mac Geek Gab guys that chown and chmod were the commands I needed to use, and that UNIX contains a pretty thorough built-in manual for almost every command. So I booted up Terminal and typed in ‘man chown’ and read up all I needed to know on the Change Owner command. I followed that with ‘man chmod’ to learn about the Change Mode command.

What I didn’t know was how to unlock a file/folder or remove an ACL in UNIX. Thus, I turned to the internet; Apple Discussions, MacRumors Forum, Mac OSX Hints all played a part in lending me a hand. I discovered the chflags command and the find command. I learned of the ‘parent/child’ relationship of shell sessions, and how sudo fits in with that scheme. I learned how to test a command using echo. Very useful. I read page after page on the correct syntax to use with these commands, I wanted to know everything I could. Messing around with UNIX is no laughing matter, one can do irreparable damage with an errant dash or wrong-case letter. I wanted to be sure what I was about to make things better, not worse.

Armed with this knowledge, I returned to Terminal but before starting my rescue mission, I wanted to do a quick test. Using the mkdir (Make Directory) command I created a new folder in my User folder. I called ‘chmod‘ to modify it’s Permissions, then tried ‘chown‘ to change the owner from ‘influxx’ to ‘adam’. With this little test complete I was ready to go.

Here’s how my test looked, don’t worry we’ll cover the details in a moment:

DrOctoPro:~ influxx$ mkdir ~/TestDir
DrOctoPro:~ influxx$ chmod 777 ~/TestDir
DrOctoPro:~ influxx$ ls -ld ~/TestDir
drwxrwxrwx 2 influxx admin 68 Jan 29 12:27 /Users/influxx/TestDir
DrOctoPro:~ influxx$ chmod 600 ~/TestDir
DrOctoPro:~ influxx$ ls -ld ~/TestDir
drw——- 2 influxx admin 68 Jan 29 12:27 /Users/influxx/TestDir
DrOctoPro:~ influxx$ chown adam:admin /Volumes/Media
DrOctoPro:~ influxx$ ls -ld ~/TestDir
drw——- 2 adam admin 68 Jan 29 12:27 /Users/influxx/TestDir

The UNIX Saviour

In order to be able to issue commands to the locked drives I had to invoke the Super User, also known as the Root User. You can turn on the Root User in the Finder by using Directory Utility but most System Admin experts really advise against it. If a Standard User account is a village peasant, and the Administrative User account is King, then the Super User is God. The Super User can go anywhere and do anything. And that amount of power in the wrong hands could be catastrophic (in personal computing terms of course, not in context to global thermo-nuclear warfare).

To invoke the Super User in Terminal you preface any commands with sudo, which will prompt you for your current admin user password. Note that, when entering your password there is no visible indication of the entered characters at all, not even dots or asterisks. So pay careful attention to what you are typing. There is also no confirmation that you are logged in, just a new line with a new prompt.

Here’s an example of how the sudo command is used, where using the ls command failed as a regular Admin User. Please note that in all the following examples, ‘DrOctoPro:~ influxx$’ is the prompt showing the name of my MacPro and my admin account name. The only command I have entered is on the line following the $ sign. Any results returned by Terminal are printed in a new line below the command.

DrOctoPro:~ influxx$ ls /Volumes/Media
-bash: cd: /Volumes/Media: Permission denied

DrOctoPro:~ influxx$ sudo ls /Volumes/Media
sudo: /var/db/sudo writable by non-owner (040775), should be mode 0700

WARNING: Improper use of the sudo command could lead to data loss or the deletion of important system files. Please double-check your typing when using sudo. Type “man sudo” for more information.

To proceed, enter your password, or type Ctrl-C to abort.

Password:
Adobe Premiere Pro Preview Files Final Cut Pro Documents
Audio Media Cache Files
Audio Render Files Render Files
Capture Scratch Stock
Encoded Files _Portfolio
DrOctoPro:~ influxx$

As you can see there is no indication of a password entered or confirmed, but sudo allowed me to see a list of the contents of that directory.

The first step I took in my recovery process was to check the properties of the Media drive by using ls (List Directory Contents). Used with the -d operand, the drive will be treated as a plain file and its attributes will be returned instead of the directory contents. The -l operand returns the Long Format for the data, offering us an extended string of information. This is how it looks:

sudo ls -ld /Volumes/Media
drw-rw-r–@ 19 admin staff 714 Jan 26 00:16 /Volumes/Media

The ls command returns some useful data, lets break it down:

  • drw-rw-r–@ This is the permissions data
  • 19 is the number of links, or hierarchical files/folders inside this directory
  • admin is the account name of the Owner
  • staff is the name of the Group the file/folder belongs to
  • 714 is the size of the folder in bytes, not including any data stored inside it
  • Jan 26 00:16 is the Date Modified data
  • /Volumes/Media is the path on my system

Looking closer at the permissions string, d indicates this is a directory. rw- means Read Access granted, Write Access granted, Execute Access denied. If Execute access had been granted it would read rwx. There are three sets of access in the string; Owner access, Group access and Everyone/Guest access. It is typical to see a string of rwxrwxr– which indicates Read/Write/Execute access for the owner and members of a Group, but everyone else is limited to Read access only. If you have seen the Permissions settings at the bottom of a files Get Info window these represent the exact same settings. Unfortunately, the GUI for setting the Permissions is not always reliable, which why we turn to the Super User to modify them. The @ at the end of this string denotes Extended Attributes, which often means the use of a custom icon or Finder comments.

permissions.jpg

In the case of my Media drive, the Owner (admin, me) has Read/Write but not Execute access. This means I am unable to open the drive in the Finder; I have the permission to read the data on it, but I have been denied the permission to open the door. That’s just the way UNIX works, and in the context of a document it would make perfect sense. Somewhere in the scheme of things my ACL for the Media drive got corrupted and Execute privileges got switched off.

Now that my suspicions had been confirmed, my task then was to use chown and chmod to return proper ownership and access to me again. Before I could do that, we recall when looking at the Get Info window the drive icon was showing a padlock. Every attempt to change permissions in the Finder were denied because this drive had somehow become locked.

To confirm that that I searched for the User Immutable Flag using the ls command again, this time with the -aOled operands, and this was the return:

ls -aOled /Volumes/Media
drw-rw-r–@ 19 admin staff uchg 714 Jan 26 00:16 /Volumes/Media

The uchg string tells me the user immutable flag changed, basically the drive has been locked. Before I could reset ownership permissions, I had to force unlock the drive. To fix that I issued a chflags command to unlock the drive:

sudo chflags nouchg /Volumes/Media

Also, I decided to search for other files on the drive that were locked. Using find will search for any file on the declared Volume that matches the declared options. The -flags uchg option will search for any file that has been flagged as locked. The -exec option will then execute the chflags command with a nouchg option and thus the User Immutable Bit is reversed, and the file is now unlocked.

sudo find /Volumes/Media -flags uchg -exec chflags nouchg {} \;

I could have used a more brute-force method on the entire contents of the drive using the -R operand of the chflags command:

sudo chflags -R nouchg /Volumes/Media

This makes the operation recursive, which means it will go down through the hierarchy of files and folders and perform the nouchg operation on everyone of them. But this will change the modify date to each and every file on the drive, causing excessively lengthy updates to Time Machine or other similar back-up software.

Another search for locked files confirms everything is now unlocked

sudo find /Volumes/Media -flags uchg

With the drive now successfully unlocked, it’s an easy task to reset ownership to myself and the admin Group

sudo chown influxx:admin /Volumes/Media

And finally reset the permissions to rwxrwxr– skipping the sudo command now that I was once again the rightful owner

chmod 755 /Volumes/Media

The entire Terminal session can be downloaded here if you want to see the order of commands I used. I have come across immovable locked files in the past, but now I am armed with this new found UNIX knowledge I am confident none of them will prove much of an obstacle. This prosess represents a few hours of research and a few minutes of actual prompting. Stumped at first, I actually only needed to type three lines of commands. I thought it worthwhile to share my entire process as it seems like a fairly typical system level encounter.

I hope you find these techniques useful, there are many more complex and useful Terminal hacks and tips on the web, this is but the tip of the iceberg. Have a try, but I don’t claim any responsibility if you sudo your drive into oblivion. Remember, with great power comes great…well you know the quote!

UPDATE

Upon some further research on the Apple Discussions, I came upon this little utility : BatChmod. I found this a little too late to help me, but even though I spent a day researching and command-lining in Terminal, I consider that time well spent as it gave me a really solid understanding of how permissions and some of the more common UNIX commands work.

Tags: , , , ,

Leave a Reply