Archive of May 2008


moving 120gb of data between servers

NOTE: this is a re-post of my original entry from my old blog here

as i mentioned previously the transfer of 120+ gb did not go smoothly. we ended up tar’ing the directory into 1 gb chunks for transfer, that way if the connection timed out we could spot exactly what file was last successfully transfered. the tar command does this nicely, however, i didn’t do it so i can’t relay how. look it up i guess.

anyway so we got the files moved, another tar command re-assembled them, and hooray! duplicated 120 gb of data cloned. next is the database…

3 databases (mysql), ~60 tables total, with 6 tables having 500,000 + rows. lots of rows.

rule 1: haste != good. read things first. when phpmyadmin offers you an export option and gives you a “maximum rows” field, make sure that value is larger than what you’re trying to export. more importantly, don’t get confused/mad/frustrated when it doesn’t work because you went through the motions and didn’t read the instructions.

rule 2: when phpmyadmin gives you an import option that is limited to 2mb files, it’s limited to 2mb files. can you change this value somewhere, in some config file? probably, i don’t know. faster than you can load the data manually with some ssh and unix skills? probably not. transfer the .sql’s to the remote server that has the database on it and type this into the shell:

mysql -u username -p table_name < sqlfile.sql

make sure username has sufficient privileges. it helps.

rule 3: documentation is your friend i know it’s a pain to read sometimes, especially when you’re in the office on a saturday, and it’s barbecue weather outside, and coffee has replaced food as fuel for your body, but it’s there for a reason. a problem arrived with php’s setup: on the old server it was running under CGI, on new server it’s embedded into apache. the web app we were running required different permissions on different files depending on the case. the difference stems from permissions: when php is embedded things are run under the “apache user”, so permissions need to be set appropriately for it to changes things. specifically folders needed to be 755, contents needed to be 644. the web app, over the course of time, has created 7,000 + folders… lots to chmod on.

fortunately Find is an excellent helper in getting this done. basically, you find things that fit criteria (imagine that!) and tell it do something with it.

Find . -type d

this should find all things of type ‘d’ (d for directory) in location ‘.’, which in unix means “right here”. so make sure you’re where you want to be, say, the top directory of what needs to be changed. find then has an “-exec” option, so you can tell it to do something with all the stuff you find

Find . -type d -exec chmod 755 {}\;

this should change every folder in the current directory (recursively) and change the permissions to 755. i can’t say what “{}\;” really does. i know chmod wants another argument (chmod 777 thing), so i guess {} means “everything” or something like that.

so to change all the contents to 644

Find . -type f -exec chmod 644 {}\;

“f” for “regular file”

so there you go, that was the fun unix / mysql stuff i did over the weekend. we repeated the process today, as today is the cutoff, and the whole process took 1/5 the time once i knew what i was doing UPDATE:

Find . -type f -exec chmod 644 '{}' \;

find is lower case. i just looked up my own blog post to use this command and it wouldn't work. also, i added quotes around the brackets and added a space between closing bracket and the \;

May 23rd, 2008

Migrating to a MacBook Air

My great aunt is traveling soon and doesn't want to lug her 17 MacBook Pro around. I can't blame her, the 17's are too big for travel IMO. Being the tech savy woman she is she picked up a new MacBook Air, smaller, lighter, more beautiful.

She knew I am a mac fan like herself, so after she un-boxed it, she re-boxed so I too could un-box it... it's been a long time since i've seen a new dell desktop/laptop un-boxed, i highly doubt it's as well designed as the mac's come.

The mission i was tasked with was migrating her user account from the mbp to the mba, excluding the iTunes library, which her's is somewhere around 25 gig. She and my great Uncle were kind enough to invite my wife and i over for dinner and drinks while the computers did their thing, they're great hosts and a lot of fun to hang out with, plus our puppy got to run around with their dog and he had a great time too. i'm getting off topic though...

Round 1

i read up on migration assistant before hand and though this couldn't be any easier, and for the most part i was right, except i ran into more than a few road blocks.

First, i fired up her external hard-drive and moved iTunes there, both as a backup and to get it off of her user account so migration assistant wouldn't try to move it.

Second, i updated the migration assistant on the mbp with the Mac OSX dvd that came with the air. With both machines on the wireless network, i ran migration assistant and got them talking. again, couldn't have been easier.

Estimated time 18 hours. Ouch. This was an older Airport Extreme, so i'm thinking 802.11b. regardless, it wasn't going to be pretty. With no switch or ethernet cords (poor planning on my part) i hit continue and we ate a delicious beer-butt chicken cooked in their BGE, with potato's and carrots and apples thrown in there too. Delicious!

After dinner and drinks we still had some 14 hours left, so we went home, me with the full intention of coming back the next day after work. Sadly, i got an email the next morning saying it hung with 15 minutes remaining...

Round 2

At work i grabbed a gigabit switch and two ethernet cords. WIFI has failed me, i'll go another route. at lunch i picked up the USB-Ethernet adaptor at a local mac shop. After work, i took 20 minutes to setup a dummy account on my laptop with a 2 gig encoded movie on the desktop and tried to transfer to my work iMac with the switch. Worked flawlessly, taking about 2 minutes.

Back at the Aunt's, i powered on the air and it went through the typical welcome to a new mac setup, NOT the "login in as mary" i was hoping to get, since only 3% didn't get transfered. So i begin hooking things up to the switch.

The adaptor worked as advertised, which was expected. I selected her account and tried to migrate, this time with the gigabit switch the ETA for transfer was 1 hour! Much, much nicer than 18.

Clicking continue, migration assistant said there was already another account by that name on the machine, and i would have to rename this new one... the plot thickens. The account was there, but i couldn't login. I figured because the initial setup of "who you are" etc hasn't been done.

After thinking for a minute, i went ahead and restarted the air, this time skipping migration assistant and setting up the laptop like new. entering their apple id, everything was already there for me and i just kept clicking "ok". i setup a dummy account and once logged in, checked the user accounts preference tab. The idea was that if her migrated account was only there 'in spirit' (meaning in the accounts list but no data, or not much), or visa-versa not in the accounts list but with data in /users, i would go into terminal and clean out the data that was laying around, and try again with migration assistant and the switch. To my surpise, there was her migrated account. i logged out of the dummy account and was able to login to her migrated account with no problems.

after looking for 5 minutes, we could only find 2 things that were not migrated, and they were on the original hard drive. everything from her user account was there, all her settings, bookmarks, everything. very nice. i opened the air up for file sharing with the original, moved those files, and done. i removed the dummy account, restored her iTunes on the mbp and everyone's happy.

So, in the end, migration assistant worked great, except it failed, kind-of-sorta. If you're going to do this, get a gigabit switch, the $30 adapter, some gin and tonic, and enjoy yourself. Wireless migration SUCKS, even if it worked.

Given my hands on time with the air, i now know for sure it's not for me. not yet. It's sooo sexy, small, light, it's amazing. but not for me right now. If i were a student or traveled a lot, it would probably be ideal, but i like having a dvd burner, and i LOVE my 15" screen. i personally think 15 is the magic number, 13 too small, 17 too big (for laptops of course). I do, however, see the air as a sign of things to come. much like the original iPod wasn't for me, i think the technology that's in that thing will set some standards for laptops. there are exciting things coming our way!

May 16th, 2008

really.png

really? ® and in the folder name? seriously, we know...

May 13th, 2008

Next →