Quick tip time! This time it’s about one of the best text editors for the mac, TextMate.
Did you know that you can recursively expand project folders in TextMate by holding down the option key (⌥) and clicking the folder to open.
Ever wanted to download all the audio files off a particular webpage? You could manually click and save each one or load up terminal and use this one-liner. Remember to only use this for sites that are allowing you do download their content free of charge; stealing is wrong!
curl -s SITEURL | grep -oie "\(http\|https\|ftp\|www\).*\.\(mp3\|aac\|wav\|m4a\|ogg\|flac\|aiff\|mp4\|wma\|ram\|ra\)" | sort | uniq | awk '{print "curl -sOL "$1" &"}' | /bin/sh
A quick explanation of what is happening.
curl -s SITEURL
This is downloading the SITEURL (which you should replace with the url of the site/file you want to scrape) but not outputting it nor showing the download progress. This is because of the -s flag that is thrown.
| grep -oie "\(http\|https\|ftp\|www\).*\.\(mp3\|aac\|wav\|m4a\|ogg\|flac\|aiff\|mp4\|wma\|ram\|ra\)"
This is parsing/scraping the site/file that we just downloaded for urls that match our regular expression. What we are looking for is urls that start with http, https, ftp, or www and that end with mp3, aac, wav, m4a, ogg, flac, aiff, mp4, wma, ram, or ra.
| sort | uniq
Sorts, and only displays the unique urls. No need for duplicates!
| awk '{print "curl -sOL "$1" &"}'
Displays the urls as an output of curl -sOL MP3URL &, which will actually download the file in the background. The -O flag will create a file and the -L flag will follow any url redirection.
| /bin/sh
Finally, the whole process is ran through the bash (shell) command.
You shouldn’t see any output because everything is ran in the background. Just wait for the files to be created in the directory you are in.
Update: The nice people at MacPorts have update the package to v1.8.0. You can follow their instructions to upgrade to the latest stable version. I would suggest the MacPorts v1.8.0 for OS X 10.6 DMG file.
After installing Snow Leopard, you may want to install the MacPorts. Follow the following steps to get it up and running:
svn co http://svn.macports.org/repository/macports/trunk/base/cd base./configuremakesudo make installsudo /opt/local/bin/port -v selfupdatesudo port upgrade --enforce-variants (Did not work for me)sudo port upgrade outdated (Did not work for me)One little problem people—including myself—have been running into is bash color codes. Many .bashrc files have the following (or similar):
export COLOR_NC='\e[0m' # No Color
export COLOR_WHITE='\e[1;37m'
export COLOR_BLACK='\e[0;30m'
export COLOR_BLUE='\e[0;34m'
export COLOR_LIGHT_BLUE='\e[1;34m'
export COLOR_GREEN='\e[0;32m'
export COLOR_LIGHT_GREEN='\e[1;32m'
export COLOR_CYAN='\e[0;36m'
export COLOR_LIGHT_CYAN='\e[1;36m'
export COLOR_RED='\e[0;31m'
export COLOR_LIGHT_RED='\e[1;31m'
export COLOR_PURPLE='\e[0;35m'
export COLOR_LIGHT_PURPLE='\e[1;35m'
export COLOR_BROWN='\e[0;33m'
export COLOR_YELLOW='\e[1;33m'
export COLOR_GRAY='\e[1;30m'
export COLOR_LIGHT_GRAY='\e[0;37m'
The solution is to use \033 as the escape code rather than \e.
# Setup some colors to use later in interactive shell or scripts
export COLOR_NC='\033[0m' # No Color
export COLOR_WHITE='\033[1;37m'
export COLOR_BLACK='\033[0;30m'
export COLOR_BLUE='\033[0;34m'
export COLOR_LIGHT_BLUE='\033[1;34m'
export COLOR_GREEN='\033[0;32m'
export COLOR_LIGHT_GREEN='\033[1;32m'
export COLOR_CYAN='\033[0;36m'
export COLOR_LIGHT_CYAN='\033[1;36m'
export COLOR_RED='\033[0;31m'
export COLOR_LIGHT_RED='\033[1;31m'
export COLOR_PURPLE='\033[0;35m'
export COLOR_LIGHT_PURPLE='\033[1;35m'
export COLOR_BROWN='\033[0;33m'
export COLOR_YELLOW='\033[1;33m'
export COLOR_GRAY='\033[1;30m'
export COLOR_LIGHT_GRAY='\033[0;37m'
I’ve always hated the fact that I would have 5-10 tabs open in Terminal, and they would all say bash.
After doing some quick research, I stumbled on decent solution.
You can add the following code block to either your .bash_profile or .bashrc.
function set_window_and_tab_title
{
local title="$1"
if [[ -z "$title" ]]; then
title="root"
fi
local tmpdir=~/Library/Caches/${FUNCNAME}_temp
local cmdfile="$tmpdir/$title"
# Set window title
echo -n -e "\e]0;${title}\a"
# Set tab title
if [[ -n ${CURRENT_TAB_TITLE_PID:+1} ]]; then
kill $CURRENT_TAB_TITLE_PID
fi
mkdir -p $tmpdir
ln /bin/sleep "$cmdfile"
"$cmdfile" 10 &
CURRENT_TAB_TITLE_PID=$(jobs -x echo %+)
disown %+
kill -STOP $CURRENT_TAB_TITLE_PID
command rm -f "$cmdfile"
}
PROMPT_COMMAND='set_window_and_tab_title "${PWD##*/}"'
Restart bash and everything should be working, hunky-dory. One problem you’ll notice is that when you try to close a session, terminal will prompt you about running processes. This is a side-affect of forking the process. A quick fix is to set the "Prompt before closing" in Preferences > Settings > Shell to “Never”.
Update: Added more data to show differences.
During the journey of finding a better linear array search function in javascript, I decided to write my own. I decided to take a different approach. Here’s the outcome.
Array.prototype.small_search = function(search) {
if (this[0] === search){ return 0; }
var s = '\x00';
var a = this.join(s);
var b = a.search(search)-1;
return a.substring(0, b).split(s).length;
};
With an array of less than 150 elements, it is much faster than Dustin Diaz’s in_array function. While they aren’t the same, his can be adapted to return the index also.
The only downside with my function is that it becomes much slower after the array length is larger than 150 elements. So, if you have a small array, go ahead and use this, otherwise use Dustin’s.
Times
Array Length: 5
small_search: 0.020ms
in_array: 0.098ms
Array Length: 10
small_search: 0.024ms
in_array: 0.084ms
Array Length: 50
small_search: 0.042ms
in_array: 0.095ms
Array Length: 100
small_search: 0.069ms
in_array: 0.106ms
Array Length: 150
small_search: 0.087ms
in_array: 0.111ms
Array Length: 300
small_search: 0.203ms
in_array: 0.137ms
Array Length: 500
small_search: 0.418ms
in_array: 0.132ms
Array Length: 1000
small_search: 0.813ms
in_array: 0.138ms
Array Length: 5000
small_search: 3.530ms
in_array: 0.312ms
Array Length: 10000
small_search: 6.718ms
in_array: 0.441ms
Test it out on your own and come back with results.
Let’s say you have a huge text file with couple million lines. Now you want to split it up into 100, 200, or 300 line files. Bash makes it pretty easy with split.
In Bash:
split -l 500 file.txt
That splits up the file file.txt into 500 line chunks. What is produced are files such as xaa, xab, xac and so on.
Be sure to do man split for more options.
Sometimes it’s very useful to know how much content is in a directory without opening a GUI interface.
In Bash:
du -cks * | sort -n | awk '\''BEGIN { split("KB,MB,GB,TB", Units, ","); } { u = 1;while ($1 >= 1024){$1 = $1 / 1024;u += 1;}$1 = sprintf("%.1f %s", $1, Units[u]);print $0;}'\'' | tail -11
I would suggest adding this to your bash aliases as ducks.
Output looks something like so:
~ > ducks 4.0 KB p 72.0 KB Music 24.1 MB Sites 35.0 MB Downloads 433.3 MB Dropbox 937.5 MB Movies 3.5 GB Library 6.3 GB Desktop 11.7 GB Documents 16.5 GB Pictures 39.5 GB total ~ > _
Update: My good friend Clayton suggested a much simpler way. The only problem is that the output is not sorted.
du -h -d 1
The output is the whole directory. I truncated the output to show the last 11.
~ > du -h -d 1 | tail -11 6.3G ./Desktop 12G ./Documents 212M ./Downloads 433M ./Dropbox 3.6G ./Library 8.0K ./Movies 72K ./Music 17G ./Pictures 0B ./Public 24M ./Sites 40G . ~ >
Update #2: Some systems do not accept the -d flag. This can be replaced with the --max-depth flag, like so:
du -h --max-depth 1
Sure you can load up a bulky editor and use its tools to do that, but why not do it in much faster in bash with awk.
In Bash:
awk '!x[$0]++' in.txt > out.txt
Wasn’t that easy?
Unicode does define some other characters which are sort of Mac-specific.
⌘ - ⌘ - ⌘ – the Command Key symbol
⌥ - ⌥ - ⌥ – the Option Key symbol
⇧ - ⇧ - ⇧ – the Shift Key symbol
⎋ - ⎋ - ⎋ – the Power Button
Note: The Power Button and Shift Key are not Mac-specific. The power button is described as “broken circle with northwest arrow”, or an escape character from ISO 9995-7. The shift key is described as an “outline up-arrow”.
Even though these are defined in standard Unicode, there is no guarantee that they will exist in the font of the receiving browser, but they’;re at least globally defined, so they’;re fair game.