Part II: Linux Boot Optimization: Results

Easily replace the init with a shell script based on /etc/rc.d/rc.sysinit and try to

tune it up as a result.

The results are pretty good I think, here is the general time line made

with a wallclock:



00: exit grub; start booting the kernel

04: kernel prints audit()

11: initrd is mounted; Red Hat nash visible

   mount / ro (normal initrd procedure)

13: start bootchart logging; start readahead of approx 193MB files

   sleep until readahead is complete

24: readahead done; now

   create /dev and modprobe (in background)

   mount / rw, enable swap

   start xfs

   startx as user davidz in background

   start messagebus

   start hald

   start acpid

   start NetworkManager

32: X claims the display

34: GNOME desktop banner

40: GNOME desktop is usable (Nautilus desktop, panel fully populated)



Here is a bootchart made with the bootchart software from Ziga Mahkovec:



http://people.redhat.com/davidz/bootchart.png



Thanks to David as I said before, I'm writing from his works here:



"You may notice that you can also start firefox after login and it starts very

very fast that's because readahead loads all files used by Firefox

in earlier experiments. I've also added files from OpenOffice.org to

readahead and that meant I could start up OpenOffice.org Writer in about

three seconds. More below.



I've made the following observations



1. The kernel patch, linux-2.6.3-printopen.patch, wasn't really working

 well for me - it reported far to few files - instead I added a

 printk() to fs/namei.c:link_path_walk()

 (disclaimer: I don't know much about the kernel so there may be a

 better solution than this).



2. The data captured from link_path_walk() was massaged into a list

 of unique files to preload and sorted on sectors.



3. While capturing the data link_path_walk() and before processing

 I went through all the menus in the GNOME desktop (to make sure

 their icon and desktop files would be added to the list) as well as

 loading Firefox. The list contains 5189 unique files - 231 of these

 from my home directory - 103 of these from gconf in my home

 directory and 302 from gconf in /etc. 2267 were .png files and

 814 of them were .desktop files. 488 files had ".so" in their name.

 There was a total of 193MB of files (which says something about

 the footprint of the GNOME desktop on Fedora :-/)



4. Doing the readahead really helped the time from startx till a

 usable desktop - less than 10 seconds!



5. Doing readahead on the 5189 files took about 45 seconds on my

 system, mostly because the files were scattered around the disk.

 Since I had a spare partition 17GB partition, I did this:

  a. format spare partition as ext3

  b. copy all readahead files to spare partition (193MB)

  c. copy rest of files from main partition to spare partition

     (about 9GB)

 Now the readahead is down to 11 seconds which averages out to

 be 18MB/s. On the other hand, I can still see (using fileblock)

 that the files in the readahead is still scattered out and hdparm

 says I should be able to get 33.87 MB/sec with no seeks.



6. I made a hack to cache /dev (a dev.tar file) and the list of modules

 to load. This could be used in production if the kernel could give

 us basically a hash value for the kobject hierarchy representing

 the hardware (perhaps even a 'tree /sys |md5sum' would suffice).

 This shaved some seconds of as well.



7. A number of things was started in parallel - I found that doing

 readahead while running modprobe wasn't helping anything; in fact

 it contributed negatively to performance (a bit to my surprise, I

 guess because the kernel was busy).



8. readahead on the right files is A Good Thing(tm). Booting my system

 without readahead on the partition with the readahead files scattered

 took 58 seconds (compared to 39 with readahead on the optimized

 partition)



  http://people.redhat.com/davidz/bootchart-without-readahead-scattered.png



 and without readahead on on the optimized partition it took 43

 seconds



  http://people.redhat.com/davidz/bootchart-without-readahead-nonscattered.png



 again compared to 39 seconds. As an added bonus, the readahead

 makes sure that e.g Firefox loads fast; all .png and .desktop files

 are in place for when using the menus. As mentioned, one could put

 very big apps like e.g. OO.o in the readahead set.



So, I think these numbers are good and there's still some room for

improvement; e.g. it takes ten seconds from grub to when the initrd is

mounted - surely the kernel can boot faster? It's after all 25% of the

time spent from grub until I have usable desktop.



The bad thing is that this approach is highly specific to my system (and

thus why I'm not posting an RPM with it :-), however I think it clearly

shows where improvements should be made; here are some random thoughts



a. We should keep track of files being loaded and maintain the

  readahead fileset as appropriate. printk() doesn't seem like the

  right solution; perhaps a system daemon using inotify or the

  kernel events layer is the road ahead? This would enable us to

  readahead the KDE stuff if the user is e.g. using KDE a lot.



b. ext3 should support operations for moving blocks around; e.g.

  optimize around the readahead fileset - when idle the system should

  rearrange the files to facilitate faster booting



c. the start_udev and kmodule process could be cached as I did above



d. The whole init(1) procedure seems dated; perhaps something more

  modern built on top of D-BUS is the right choice - SystemServices

  by Seth Nickell comes to mind [1]. Ideally services to be started

  would have dependencies such as 1) don't start the gdm service

  before /usr/bin/gdm is available; 2) the SSH service would only

  be active when NetworkManager says there is a network connection;

  /usr from LABEL=/usr would only be mounted when there is a volume

  with that label and so forth. Also, such a system would of course

  have support for LSB init scripts.

  (This is probably a whole project on it's own so I'm omitting

  detailed thinking on it for now)

"

Thanks a lot to Ziga Mahkovec for the bootchart software - it's been

very useful.

Part I: Linux Boot Optimization: Introduction & Implementation thought

... somehow I was thinking about boot processes and its time and payloads on Windows and Linux and compare their "boot time" and the time that the OS ready to respond to a request from a network, console or so on.Thanks to David Zeuthen & Owen Taylor and after of all tests there were so many usefull results which I would like to brief them.

Currently, the time to boot the Linux desktop from the point where the power switch is turned on, to the point where the user can start doing work is roughly two minutes.
During that time, there are basically three resources being used: the hard disk, the CPU, and the natural latency of external systems - the time it takes a monitor to respond to a DDC probe, the time it takes for the system to get an IP via DCHP, and so forth.
Ideally, system boot would involve a 3-4 second sequential read of around 100 megabytes of data from the hard disk, CPU utilization would be parallelized with that, and all queries on external systems would be asynchronous ... startup continues and once the external system responds, the system state is updated. Plausibly the user could start work under 10 seconds on this ideal system.
The challenge is to create a single poster showing graphically what is going on during the boot, what is the utilization of resources, how the current boot differs from the ideal world of 100% disk and CPU utilization, and thus, where are the opportunities for optimization.

So had a brief look at shortening startup/login time and tried

disabling rhgb in favor of starting gdm early. It looks pretty

promising; here are some wall-clock numbers from two runs of each

configuration:



                    |   gdm_early  |   rhgb+gdm   |

----------------------+------+-------+-------+------+

GRUB timeout          | 0:00 | 0:00  |  0:00 | 0:00 |

Starting udev         | 0:13 | 0:13  |  0:13 | 0:14 |

HW init done          | 0:25 | 0:25  |  0:26 | 0:26 |

rhgb visible          |  N/A |  N/A  |  0:36 | 0:35 |

gdm login visible     | 0:43 | 0:44  |  1:25 | 1:26 |

gdm login entered     | 0:52 | 0:52  |  1:31 | 1:32 |

GNOME banner visible  | 1:13 | 1:14  |  1:40 | 1:41 |

Nautilus Background   | 1:33 | 1:32  |  1:51 | 1:52 |

Panel visible         | 1:43 | 1:43  |  2:02 | 2:02 |

HD activity off       | 1:59 | 1:56  |  2:13 | 2:14 |



The milestones should be pretty self evident. This is on a stock FC3

system running on a IBM T41 1.6GHz (running on AC power), 512MB RAM

without any services manually disabled.



In addition to starting gdm early, the modifications also start up a few

services, D-BUS, HAL and NetworkManager, that is critical to the GNOME

desktop.



Some random thoughts/observations:



- We get the gdm window 40 secs faster



- The 12 secs from "Starting udev" to "HW init done" can be mostly

shaved away/run in parallel



- Kernel bootstrap time (13 secs) can probably be much shorter

(that's what some kernel guys say anyway)



- With this hack we shave twenty secs of the booting time (e.g. from

GRUB until you can use your PC) but booting still feels much quicker

because of the interaction with gdm in the middle (YMMV; e.g. placebo

effect etc.)



- rhgb+gdm spawns an X server each which is sort of stupid and unsafe

(or so some Xorg guys tell me). This solution, per design, avoids

doing that



- we don't get the kudzu screen nor the fsck screens or any other

console interactions. However, IMHO, such screens are not good UI

in the first place - we should instead have GUI replacemnts that

possibly notifies you when you log into the desktop session (stuff

like NetworkManager and HAL alleviates such problems for networking

and storage devices)



- we don't get service startup notification, but, uhmm, is it really

useful learning that the "Console Mouse Service" or "Printing Sub-

system" have started? Instead, this stuff could just be put in gdm



- it could be interesting to make /sbin/init own a D-BUS service that

gdm and other stuff can query and interact with. Could also be fun

to completely replace it with something a'la the SystemServices

prototype that Seth did last year; links



http://www.osnews.com/story.php?news_id=4711

http://www.gnome.org/~seth/blog/2003/Sep/27



- Could be interesting to instrument the kernel with some pagefault

counters etc. and attempt do more readahead on e.g. the GNOME libs

(both Windows XP and Mac OS X does all that; I think we do too but

I've been told it can be improved)



So, anyway, I think it could be interesting to discuss starting gdm

instead of rhgb. If you want to try out my crude hack, grab the file

here



http://people.redhat.com/davidz/newinit.sh



put it in on your system as /newinit.sh, chmod a+x it and change this

line /etc/inittab



si::sysinit:/etc/rc.d/rc.sysinit



to these two lines



#si::sysinit:/etc/rc.d/rc.sysinit

si::sysinit:/newinit.sh



and you should be set to go! If it breaks you get to keep both pieces;

e.g. try this at your own risk [1].

IE "Save Picture As..." Image Download Spoofing

A vulnerability in Microsoft Internet Explorer has been discovered, which can be exploited by malicious people to trick users into downloading malicious files. The vulnerability is caused due to Internet Explorer using the file extension from the URL’s filename when saving images with the “Save Picture As” command and also strips the last file extension if multiple file extensions exist. This can be exploited by a malicious web site to cause a valid image with malicious, embedded script code to be saved with an arbitrary file extension. Successful exploitation may allow a malicious web site to trick users into downloading e.g. a malicious HTML Application (.hta) masqueraded as a valid image. However, exploitation requires that the option “Hide extension for known file types” is enabled (default setting). The vulnerability has been confirmed on a fully patched system with Internet Explorer 6.0 and Microsoft Windows XP SP2.
Solution : Disable the “Hide extension for known file types” option.

How to help protect against a WINS security issue

Microsoft is investigating reports of a security issue with Microsoft Windows Internet Name Service (WINS). This security issue affects Microsoft Windows NT Server 4.0, Microsoft Windows NT Server 4.0 Terminal Server Edition, Microsoft Windows 2000 Server, and Microsoft Windows Server 2003. Microsoft Windows 2000 Professional, Microsoft Windows XP, and Microsoft Windows Millennium Edition are not affected by this vulnerability.
By default, WINS is not installed on Windows NT Server 4.0, on Windows NT Server 4.0 Terminal Server Edition, on Windows 2000 Server, or on Windows Server 2003. By default, WINS is installed and running on Microsoft Small Business Server 2000 and on Microsoft Windows Small Business Server 2003. However, by default, on all versions of Microsoft Small Business Server, the WINS component communication ports are blocked from the Internet, and WINS is available only on the local network.
This security issue could make it possible for an attacker to remotely take control of a WINS server. As of November 26, 2004, Microsoft is not aware of any customers who have been affected by this security issue. Microsoft will continue to investigate this security issue to determine the appropriate steps to help protect the customers. Additionally, to help protect your computer against this security issue, follow the steps in http://support.microsoft.com/default.aspx?scid=kb;en-us;890710

Some Net/FreeBSD/Linux Redhat Kernel Tunnig Variables

This summary is not available. Please click here to view the post.

Cool Websites about Data Storage

Cool Websites about Data Storage:

• Recover Data - Data Internet Raid Recovery

• Data - Oracle Data Warehousing

• Recover Data - Data Recover Software

• Recover Data - Data Security Systems

• Database Storage - Data Storage On The Web

• Data Recovery - Digital Image Storage Device

Hackers Taking Aim at Mobile Phones

Be aware!Cell phones are becoming frequent targets for viruses, spam and other malicious attacks because most are not programmed with protective software.
- The Washington Post