Difference between revisions of "Aws-create"

__NOTOC__

= Creating a CentOS AMI through a Loopback File =

The image created here is based on CentOS Linux.

== Create a File to Host the CentOS AMI ==

A base CentOS Linux installation requires about 1.2 GB, but custom installation for a server environment might require up to 3.0 GB.

* Create a location to place the CentOS AMI

<pre><nowiki>

# mkdir -p /usr/local/src/EC2

</nowiki></pre>

* Create a file to host the CentOS AMI

<pre><nowiki>

# dd if=/dev/zero of=/usr/local/src/EC2/bashlinux-server.fs bs=1M count=3072

3072+0 records in

3072+0 records out

3221225472 bytes (3.2 GB) copied, 118.687 seconds, 27.1 MB/s

</nowiki></pre>

== Create a Root File System Inside the File ==

Create an ''ext3'' file system

<pre><nowiki>

mke2fs -F -j /usr/local/src/EC2/bashlinux-server.fs

mke2fs 1.39 (29-May-2006)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

393216 inodes, 786432 blocks

39321 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=805306368

24 block groups

32768 blocks per group, 32768 fragments per group

16384 inodes per group

Superblock backups stored on blocks:

32768, 98304, 163840, 229376, 294912

Writing inode tables: done

Creating journal (16384 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 30 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

</nowiki></pre>

== Mount the File through Loopback ==

# Create a mount point in the file system where the image will be attached:

<pre><nowiki>

# mkdir -p /mnt/EC2-FS

</nowiki></pre>

# Mount the file system image:

<pre><nowiki>

mount -o loop /usr/local/src/EC2/bashlinux-server.fs /mnt/EC2-FS/

</nowiki></pre>

== Prepare for the Installation ==

# Create a `/dev` directory and populate it with a minimal set of devices. You can ignore the errors in the output

<pre><nowiki>

# mkdir -p /mnt/EC2-FS/dev

# MAKEDEV -d /mnt/EC2-FS/dev -x console

# MAKEDEV -d /mnt/EC2-FS/dev -x null

# MAKEDEV -d /mnt/EC2-FS/dev -x zero

</nowiki></pre>

# Create the `/etc` directory

<pre><nowiki>

# mkdir -p /mnt/EC2-FS/etc

</nowiki></pre>

# Create the `fstab` file within the `/mnt/EC2-FS/etc` directory and add the following:

<pre><nowiki>

/dev/sda1 / ext3 defaults 1 1

none /dev/pts devpts gid=5,mode=620 0 0

none /dev/shm tmpfs defaults 0 0

none /proc proc defaults 0 0

none /sys sysfs defaults 0 0

</nowiki></pre>

# Create a temporary yum configuration file `/usr/local/src/EC2/yum-ec2.conf`

<pre><nowiki>

[base]

name=CentOS-5.1 - Base

mirrorlist=http://mirrorlist.centos.org/?release=5.1&arch=i386&repo=os

baseurl=http://mirror.centos.org/centos/5.1/os/i386/

gpgcheck=0

[updates]

name=CentOS-5.1 - Updates

mirrorlist=http://mirrorlist.centos.org/?release=5.1&arch=i386&repo=updates

baseurl=http://mirror.centos.org/centos/5.1/updates/i386/

gpgcheck=0

</nowiki></pre>

This ensures all the required basic packages and utilities are installed. This file can be located on image location `/usr/local/src/EC2`, and is used only during installation

# Hide system repositories for AMI installation

<pre><nowiki>

# mkdir /root/tmp-yum

# mv /etc/yum.respos.d/* /root/tmp-yum/

</nowiki></pre>

This avoids the use of system repositories that requires a specific value for `$releasever` and `$basearch`, also avoids the duplicated repository error messages.

# Mount the `proc` filesystem manually

<pre><nowiki>

# mkdir -p /mnt/EC2-FS/proc

# mount -t proc none /mnt/EC2-FS/proc

</nowiki></pre>

# Create log folder

<pre><nowiki>

# mkdir -p /mnt/EC2-FS/var/log

</nowiki></pre>

== Install the Operating System ==

* Install the Base operating system

<pre><nowiki>

# yum -c /usr/local/src/EC2/yum-ec2.conf --installroot=/mnt/EC2-FS --exclude=sysreport -y groupinstall Core Base

Core Base

Repository base is listed more than once in the configuration

Repository updates is listed more than once in the configuration

Setting up Group Process

Setting up repositories

extras 100% |=========================| 1.1 kB 00:00

updates 100% |=========================| 951 B 00:00

base 100% |=========================| 1.1 kB 00:00

addons 100% |=========================| 951 B 00:00

yumgroups.xml 100% |=========================| 9.7 kB 00:00

comps.xml 100% |=========================| 912 kB 00:05

Setting up repositories

Reading repository metadata in from local files

primary.xml.gz 113 kB 00:02

################################################## 421/421

primary.xml.gz 100% |=========================| 278 kB 00:01

################################################## 502/502

primary.xml.gz 100% |=========================| 834 kB 00:06

################################################## 2400/2400

primary.xml.gz 100% |=========================| 157 B 00:00

Excluding Packages in global exclude list

...

Complete!

</nowiki></pre>

Due compatibility errors with `sos` package, `sysreport` package should be excluded.

* Install the packages and files needed for a server

To do a fast installation try to use the "CentOS 5.1 Final" DVD and the CentOS-Media repository locate in the temporary location we set at `/root/tmp-yum`. To have done this do as follows:

# Insert "CentOS 5.1 Final" DVD

# If the sytem didn't mount automatically the DVD under `/media` folder, then run the command:

<pre><nowiki>

# mount /dev/cdrom /media/CentOS_5.1_Final

</nowiki></pre>

# Install the rest of the packages using the DVD as repository:

<pre><nowiki>

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall mysql editors system-tools text-internet gnome-desktop

yum --installroot=/mnt/EC2-FS clean all

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall ftp-server

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall network-server

yum --installroot=/mnt/EC2-FS clean all

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall java

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall java-development

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall base-x

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall graphics

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall web-server

yum --installroot=/mnt/EC2-FS clean all

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall smb-server

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall mail-server

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall server-cfg

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall sound-and-video

yum --installroot=/mnt/EC2-FS clean all

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall admin-tools

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall development-tools

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS --exclude=sysreport --exclude=nautilus-sendto -y groupinstall graphical-internet

yum -c /root/tmp-yum/CentOS-Media.repo --installroot=/mnt/EC2-FS -y install mod_auth_mysql php-mysql vim-X11 festival createrepo audit mrtg amanda-client iptraf lynx kexec-tools device-mapper-multipath vnc-server xorg-x11-server-Xnest xorg-x11-server-Xvfb libsane-hpaio tomcat5-admin-webapps php-pear tomcat5-webapps mod_auth_mysql tomcat5 dnsmasq ruby ruby-rdoc ruby-devel ruby-ri ruby-docs mysql-devel php-mysql php-bcmath php-gd php-mcrypt php-soap php-xml imake memtest86+ compat-libstdc++-33 compat-libstdc++-296 yum-cron libmcrypt

</nowiki></pre>

Due compatibility issues with _glib_ package, _nautilus-sendto_ package should be excluded.

<!-- # Install uWink packages using uWink repository -->

{noformat}

yum -c /root/tmp-yum/uWink-Base.repo --installroot=/mnt/EC2-FS2 -y install uwink-puppet uwink-system uwink-other uwink-api uwink-menu uwink-volante

{noformat}

* Restore yum repositories*

{noformat}

<!-- # mv /root/tmp-yum/* /etc/yum.repos.d/ -->

{noformat}

h2. Configure the Operating System

After successfully installing the the base operating system, you mus configure the networking and hard drives to work in the Amazon EC2 environment.

* Configure the operating system*

<!-- # Create _/mnt/EC2-FS/etc/sysconfig/network-scripts/ifcfg-eth0_ and make sure it contains at least the following information: -->

{noformat}

DEVICE=eth0

BOOTPROTO=dhcp

ONBOOT=yes

TYPE=Ethernet

USERCTL=yes

PEERDNS=yes

IPV6INIT=no

{noformat}

{note:title=Note}The Amazon EC2 DHCP server ignores hostname requests. If you set _DHCP_HOSTNAME_, the local hostname will be set on the instance but not externally. Additionally, the local hostname will be the same for all instances of the AMI, which might be confusing.

{note}

<!-- # Ensure that networking starts by making sure the following line appears in the _/mnt/EC2-FS/etc/sysconfig/network_ file: -->

{noformat}

NETWORKING=yes

{noformat}

<!-- # Create _/etc/hosts_ file with default configuration -->

{noformat}

<!-- # echo "127.0.0.1 localhost.localdomain localhost" > /mnt/EC2-FS/etc/hosts -->

{noformat}

<!-- # Ensure that SELinux is disabled -->

{noformat}

<!-- # perl -pi -e 's/SELINUX=enforcing/SELINUX=disabled/' /mnt/EC2-FS/etc/selinux/config -->

{noformat}

<!-- # Ensure that local disk storage on _/dev/sda2_ and swap space on _/dev/sda3_ are mounted at system startup by adding the following lines to _/mnt/EC2-FS/etc/fstab_: -->

{noformat}

/dev/sda2 /mnt ext3 defaults 0 0

/dev/sda3 swap swap defaults 0 0

{noformat}

{note:title=Note}The _/dev/sda2_ and _/dev/sda3_ storage locations only apply to small instances. For more information on instance storage, see [http://docs.amazonwebservices.com/AWSEC2/2008-02-01/DeveloperGuide/instance-storage.html]

{note}

<!-- # Change base root to _/mnt/EC2-FS{_}{noformat} -->

<!-- # chroot /mnt/EC2-FS -->

{noformat}

<!-- # Make sure all of your required services start at system startup by allocating them appropriate system run levels: -->

{noformat}

<!-- # chkconfig --level 2345 sshd on -->

<!-- # chkconfig --level 345 monit -->

{noformat}

<!-- # In order to setup MySQL first startup the network, then fire up MySQL -->

{noformat}

<!-- # /etc/init.d/network start -->

<!-- # /etc/init.d/mysqld start -->

{noformat}

<!-- # Run post-install scripts, create a file on _/root/ks-post.sh_ and add the following commands: -->

{code:java|bgColor=#FFFFCE}

<!-- #!/bin/bash -->

<!-- # Setup user group and passwords -->

cat >> /etc/passwd << 'EOF'

brent:x:501:501:brent:/home/brent:/bin/bash

rizwank:x:502:502:rizwank:/home/rizwank:/bin/bash

steve:x:503:503:steve:/home/steve:/bin/bash

manuel:x:504:504::/home/manuel:/bin/bash

EOF

cat >> /etc/shadow << 'EOF'

brent:$1$5jZxoSH.$2Iq7IKFN3J3Ub5uidQ2Oz0:13929:0:99999:7:::

rizwank:$1$79SO1kRt$VLH/kiLyhrN9JuGDq7bAJ/:13929:0:99999:7:::

steve:$1$.XHDDPhz$4J1LkjQoqDeLk1grLMlSN.:13929:0:99999:7:::

manuel:$1$5WLy6Uzj$YmXXpbAKfCN2Q3SSSa3F91:14043:0:99999:7:::

EOF

cat >> /etc/group << 'EOF'

brent:x:501:

rizwank:x:502:

steve:x:503:

manuel:x:504:

EOF

cat >> /etc/gshadow << 'EOF'

brent:!!::

rizwank:!!::

steve:!!::

manuel:!::

EOF

<!-- # Home dirs -->

mkdir -p /home/brent

mkdir -p /home/rizwank

mkdir -p /home/steve

mkdir -p /home/manuel

chown -R 501:501 /home/brent

chown -R 502:502 /home/rizwank

chown -R 503:503 /home/steve

chown -R 504:504 /home/manuel

<!-- # Add users to sudoers -->

cat >> /etc/sudoers << 'EOF'

brent ALL=(ALL) ALL

rizwank ALL=(ALL) ALL

steve ALL=(ALL) ALL

manuel ALL=(ALL) ALL

EOF

JAVA_HOME=/usr/java/jre1.6.0_05

/usr/sbin/alternatives --install /usr/bin/java java ${JAVA_HOME}/bin/java 100 \

--slave /usr/bin/rmiregistry rmiregistry ${JAVA_HOME}/bin/rmiregistry \

--slave /usr/share/man/man1/rmiregistry.1 rmiregistry.1 ${JAVA_HOME}/man/man1/rmiregistry.1 \

--slave /usr/share/man/man1/rmiregistry.1.gz rmiregistry.1.gz ${JAVA_HOME}/man/man1/rmiregistry.1.gz

alternatives --set java ${JAVA_HOME}/bin/java

<!-- ##################################### -->

cat > /root/configure.sh << 'EOF'

<!-- #!/bin/sh -->

echo "Setting server up as node: $1"

hostname $1

echo "export NODE=$1" >> /etc/profile

IPADDR=`ifconfig eth0 | grep "inet addr" | awk '{ print $2}' | tr -d "addr:"`

echo "$IPADDR $1.uwink.com $1" >> /etc/hosts

echo "$IPADDR local.uwink.com local" >> /etc/hosts

echo "$1" > /etc/hostname

<!-- #TODO HOSTNAME might not be in sysconfig/network -->

perl -pi -e 's/HOSTNAME=.*/HOSTNAME=$1/g' /etc/sysconfig/network

EOF

chmod +x /root/configure.sh

<!-- ##################################### -->

<!-- # Database settings -->

echo "Setting up database"

mysqladmin create bistro

mysql -e "GRANT ALL ON bistro.* TO 'root'@'localhost' IDENTIFIED BY 'root'"

mysql -u root -proot -e "GRANT SELECT,UPDATE,INSERT ON bistro.* TO 'etluser'@'dsl093-116-242.lax1.dsl.speakeasy.net' IDENTIFIED BY '34td4t4' REQUIRE SSL"

mysql -u root -proot -e "GRANT SELECT,UPDATE,INSERT ON bistro.* TO 'etluser'@'eotu.uwink.com' IDENTIFIED BY '34td4t4' REQUIRE SSL"

mysql -u root -proot -e "GRANT SELECT,UPDATE,INSERT ON bistro.* TO etluser@'10.%' identified by '34td4t4' REQUIRE SSL"

mysql -u root -proot -e "GRANT ALL ON bistro.* TO 'root'@'10.%' IDENTIFIED BY '34td4t4'"

mysql -u root -proot -e "GRANT ALL ON bistro.* TO 'root'@'local.uwink.com' IDENTIFIED BY '34td4t4'"

mysqladmin --user=root --password=root flush-privileges

<!-- # Media directory settings -->

mkdir -p /opt/uwink/media

mkdir -p /opt/uwink/apps

mkdir -p /opt/uwink/archive

mkdir -p /opt/uwink/share/customerphotos

mkdir -p /var/log/uwink

ln -s /opt/uwink/media /var/www/html/media

ln -s /opt/uwink/apps /var/www/html/apps

chown bistro /opt/uwink/media

<!-- # php.ini memory_limit to 128 -->

perl -pi -e 's/memory_limit = \d*M/memory_limit = 128M/' /etc/php.ini

<!-- #Set UTF-8 as the default encoding -->

perl -pi -e 's/;mbstring.internal_encoding = EUC-JP/mbstring.internal_encoding = UTF-8/' /etc/php.ini

<!-- # OFF for test network, on for restaurants -->

<!-- #TODO remove this ultimately -->

chkconfig --level 345 actionqueue off

service actionqueue stop

<!-- #set evil symlinks for volante -->

<!-- #ln -s /usr/lib/libstdc++-libc6.2-2.so.3 /usr/lib/libstdc++-libc6.1-1.so.2 -->

<!-- #configure vncserver... TODO shouldn't be root -->

perl -pi -e 's/# VNCSERVERS="2:myusername"/VNCSERVERS="1:root"/' /etc/sysconfig/vncservers

<!-- #add our favicon -->

wget http://www.uwink.com/images/favicon.ico -O /var/www/html/favicon.ico

<!-- # Disable asking to accept ssh key -->

echo -e "\tStrictHostKeyChecking no" >> /etc/ssh/ssh_config

<!-- #Setup emailing to admin -->

perl -pi -e "s/^#root:\s*marc/root:\t\tadmin\@uwink.com/" /etc/aliases

newaliases

<!-- #TODO activate monit in proper run levels and turn on -->

<!-- #chkconfig --level 345 monit on -->

<!-- #service monit start -->

<!-- #until puppet goes into KS, allowing updates to happen is dangerous -->

rm /etc/cron.daily/yum.cron /etc/cron.weekly/yum.cron

<!-- #prep puppet config. Doesn't handle hostname yet -->

perl -pi -e 's/#PUPPET_SERVER=puppet/PUPPET_SERVER=puppet.uwink.com/' /etc/sysconfig/puppet

perl -pi -e 's/#PUPPET_LOG/PUPPET_LOG/' /etc/sysconfig/puppet

{code}

<!-- # Stop services and umount the image: -->

{noformat}

<!-- # /etc/init.d/mysqld stop -->

<!-- # /etc/init.d/network stop -->

<!-- # umount /mnt/EC2-FS/proc -->

<!-- # umount -d /mnt/EC2-FS -->

{noformat}