Data Recovery Open Source Tool!!

http://www.cgsecurity.org/wiki/TestDisk

Solid State Drives vs Hard disk drives

Intro

• A solid state drive stores its data in solid-state memory (Flash / SRAM / DRAM)
• Flash does not require constant power and is non-volatile while SRAM and DRAM are volatile

Speeds

• Flash maybe slower than even traditional HDDs on big file access
• Flash is considerably slower than conventional disks for small writes. This is partly due to their large erase block size of 0.5-1 MB
• SSDs are faster than HDDs for small random reads due to negligible seek time (no moving parts)
• Check the comparison table at http://www.storagesearch.com/ssd-ram-v-flash.html When Flash based SSDs are used for equal reads and writes they are actually slower than HDDs. However if small random reads far outweigh writes, the performance gains can be upto 100x!!?
• Download the paper - Comparison of Drive Technologies for High-Transaction Databases. Findings below -
  • HDDs: Small reads - 175 iops/s, Small writes - 280 iops/s
  • Flash SSDs: Small reads - 1075 iops/s (6x), Small writes - 21 iops/s (0.1x)
  • DRAM SSDs: Small reads - 4091 iops/s (23x), Small writes - 4184 iops/s (14x)
• Another whitepaper on Flash vs HDDs is Understanding Flash SSD Performance. Findings below -
  • Read performance: Flash outperforms hdds by a large magnitude for small block size
  • It is with write performance that Flash SSDs become problematic. The issue here is the internal structure used within the Flash storage array. This structure includes a collection of bytes called an "erase block". When you write to a Flash SSD, the drive itself cannot just update the sectors you are changing, but must merge your changes with existing data to update a complete erase block. As Flash SSDs have gotten faster and larger, erase blocks have grown as well. Flash erase blocks used to be 16K in length. Now they are 1 Megabyte for small SSDs extending up to as large as 4 Megabytes for some models.
  • If you are doing pure reads, a Flash SSD will typically be 20x faster than a hard disk for small random reads. If you are doing pure random writes, the same drive might be 15x slower than a hard disk
  • Of pertinence is the table which shows how a small % of writes can destroy Flash SSD Performance. It is for this reason alone that Flash SSDs, by themselves, are not very effective with random update applications like on-line databases, mail queues, and other environments that involve a lot of small updates
    
• One can improve write performance of a Flash SSD using the following methods -
  • OS Write caching - OS buffers writes which eventually get written to disk making the writes appear faster
  • File systems optimized to minimize random writes - YAFFS, JFFS2.
  • Managed Flash Technology - a patent pending technology by easyco which enables Flash Drives to write clusters of random data in linear streams

Costs

• As of mid-2008, SSD prices are still considerably higher per gigabyte than are comparable conventional hard drives: consumer grade drives are typically US$2.00 to US$3.45 per GB for flash drives and over US$80.00 per GB for RAM-based compared to about US$0.12 per gigabyte for hard drives
• DRAM based SSD require more power than hard disks when operating, and need continuous power when not in use if the data needs to be persistent
• Check article Flash vs DRAM Price Projections - for SSD Buyers
  • In the first half of 2007 the difference in user price between a RAM versus Flash SSD was about 45 to 1. A year later in the first half of 2008 that ratio had changed to 25 to 1
  • However NAND has been on a steeper price decline than DRAM for its entire existence. The price of a gigabyte of DRAM declines (on average) 32% per year. There are indications that this decline may slow. Meanwhile, NAND's price per gigabyte declines faster, at an average of 50% per year

My Conclusions

• DRAM based SSDs are crazy expensive. They serve best for volatile caches (eg, memcached pools etc). If you have servers dedicated to serve in-memory cache data, it may reduce your cost to add DRAM SSDs to these clusters since they are likely not going to bottle-neck on CPU anyways
• Flash based SSDs would work in an environment where the % of writes is low. As can be seen in some of the above benchmarks, a flash based SSD starts degrading in performance in comparison to HDDs in environments with just 5% writes. If one wants to use Flash based SSDs in environments with substantial writes, one should use special filesystems (YAFFS / JFFS2)  and/or use Managed Flash Technology
• Flash based SSDs work like a charm in a read-only or mostly read environment

Fundamental Concepts of Exchanging Emails On lInux

Basics of Mail Exchanging

E-Mail:

You can send message from one computer to another computer using electronic-mail around the world. E-mail is more powerful and more usable tools in today’s world. This E-mail is transfer via into the computer and the computer network. There are many Server are used to exchange these e-mail among the computer to computer and the one network to different network. The mail server is responsible or used to exchange this mail that means a mail server receive or store an e-mail message from client and deliver it to the other client. In this tutorial you will show that how an e-mail message is transfer from sender to receiver and you will also know that what happen during the time of exchanging the mail.

How a Mail Server Works:

Before we begin to configure the Linux Mail Server we need to understand that how an E-Mail system is work. An E-mail system is build-up some element which is described below:

Mail User Agent (MUA)

It works in user machine which is directly run by a user. This is used to compose e-mail message and send it to mail server or receive the mail message from server. It Just an application, such as Outlook, Thunderbird, Pine (used in Linux).

Mail Transfer Agent (MTA)

Mail Transfer Agent is used to transfer message between the machines. After receiving the mail from MUA, Mail Transfer Agent starts its work. In Red Hat Linux the default Mail Transfer Agent isSendmail or you can also use Postfix to more secure your mail server. In Unix the MTA is qmail.When an email is sent, the message is routed from server to server, all the way to the recipient's email server. After receiving the mail from MUA, MTA read the receiver address from the header part of the mail and find out the receiver server IP address, then MTA try to communicate with the 25 No Port of Server IPs of receiver . If the sender server MTA could establish the connection with receiver server MTA then sender server handover the mail of receiver server MTA using the Simple Mail Transfer Protocol (SMTP).

Mail Delivery Agent (MDA)

MDA/LDA is Mail Delivery Agent or Local Delivery Agent. Both are essentially synonymous. (Actually there are subtle differences between the two).  MDA receive the message from Mail Transfer Agent and lace into the user mailbox. In Red Hat Linux MDA/LDA is procmail.

Simple Mail Transfer Protocol (SMTP)

The SMTP is responsible for transferring the mail message from one MTA to another MTA. That means this is used to transfer mail between the computers in network. It can transfer only ASCII text. It can’t handle font, color, graphics, or attachment with messages. For this you can use MIME (Multi-purpose Internet Mail Extensions or Multimedia Internet Mail Extensions). It’s an encoding protocol like BinHex in Mac and UUEncode in UNIX. That support font, color, graphics, or attachment.  At first it was used as a way of sending more than just text via email. Later the protocol was extended to manage file typing by Web servers. MUAs and MTAs use this protocol for sending e-mails.

Post Office Protocol (POP3)

POP3 stands for a Post Office Protocol version 3. POP3 is a client/server protocol. The POP3 protocol is designed to allow the users to retrieve e-mail messages when they are connected to the email server (via Internet, Ethernet or VPN network connection). Once the email messages are downloaded from the server they can be modified, read and manipulated offline. MUAs can use this protocol to sen and receive e-mails from the server.

Internet Message Access Protocol (IMAP)

The IMAP (Internet Message Access Protocol) is a newer and modern alternative to the POP3 protocol. Unlike POP3, the IMAP allows the users to work with their messages in both online and offline modes. The IMAP-capable email client programs retrieve the messages' headers from the server and can store local copies of the messages in a local (temporary) cache. All the messages are left on the server until they are deleted by the user. This mechanism allows multiple email clients to access a single mailbox and is often used for corporate / business e-mails. MUAs can use this protocol to send and receive e-mails on the server.

Mail-Boxes (Inbox)

A mail-box is container or directory of files, where incoming messages are stored.

Server Configure

In this tutorial you will see that how to configure mail server in a sub domain DNS server namedmail.mydomain.com who’s FQDN is ns3.mail.mydomain.com that I have already created. You can also configure the mail server under your main DNS domain that will describe later. So let’s start.

Package required for mail server:  Server Side Packages

Package Name	Description
Sendmail	Default MTA In Red Hat/Fedora Linux
m4	Known as a Macro Processor, used to create sendmail.cf file
Postfix	Another MTA In Linux, used to Exchange secure e-mail
Dovecot	A Package for  Accessing the mailbox
POP3	It’s a Protocol to retrieve the e-mail messages from server
IMAP	It’s an alternative Protocol of POP3
SMTP	Used to transfer e-mail message from one MTA to another MTA
Squirrelmail	A Web Mail Configuration Tool
Php	It’s a scripting language for web development to produce dynamic web pages
Mysql	The most popular Open Source SQL database management system
System-switch-mail	Graphical Tools for Mail Transport Agent Switcher

Client Side Software

Software	Description
Outlook Express	POP3- and IMAP-compatible mail client with a built-in newsreader.
Thunderbird	It’s a graphical email client and newsreader developed by the Mozilla Foundation.
Pine	It’s an application for sending and receiving email in Linux/Unix
Mozilla Firefox, Internet Explorer, Opera, Safari, Or Any Suitable Browser.	Will be used to access mailbox using webmail.

Required configuration file for sendmail:

Sendmail is the default Mail Transfer Agent (MTA) in FreeBSD. sendmail's job is to accept mail from Mail User Agents (MUA) and deliver it to the appropriate mailer as defined by its configuration file. sendmail can also accept network connections and deliver mail to local mailboxes or deliver it to another program.

                sendmail uses the following configuration files:

File Name                                                                                          Function

/etc/mail/access                                                                      sendmail access database file

/etc/mail/aliases                         Mailbox aliases

/etc/mail/local-host-names                Lists of hosts sendmail accepts mail for

/etc/mail/mailer.conf                     Mailer program configuration

/etc/mail/mailertable                     Mailer delivery table

/etc/mail/sendmail.cf                     sendmail master configuration file

/etc/mail/virtusertable                   Virtual users and domain tables

MySql Commands!!!

This is a list of handy MySQL commands that I use time and time again. At the bottom are statements, clauses, and functions you can use in MySQL. Below that are PHP and Perl API functions you can use to interface with MySQL. To use those you will need to build PHP with MySQL functionality. To use MySQL with Perl you will need to use the Perl modules DBI and DBD::mysql.
Below when you see # it means from the unix shell. When you see mysql> it means from a MySQL prompt after logging into MySQL.
To login (from unix shell) use -h only if needed.

# [mysql dir]/bin/mysql -h hostname -u root -p
Create a database on the sql server.

mysql> create database [databasename];
List all databases on the sql server.

mysql> show databases;
Switch to a database.

mysql> use [db name];
To see all the tables in the db.

mysql> show tables;
To see database's field formats.

mysql> describe [table name];
To delete a db.

mysql> drop database [database name];
To delete a table.

mysql> drop table [table name];
Show all data in a table.

mysql> SELECT * FROM [table name];
Returns the columns and column information pertaining to the designated table.

mysql> show columns from [table name];
Show certain selected rows with the value "whatever".

mysql> SELECT * FROM [table name] WHERE [field name] = "whatever";
Show all records containing the name "Bob" AND the phone number '3444444'.

mysql> SELECT * FROM [table name] WHERE name = "Bob" AND phone_number = '3444444';
Show all records not containing the name "Bob" AND the phone number '3444444' order by the phone_number field.

mysql> SELECT * FROM [table name] WHERE name != "Bob" AND phone_number = '3444444' order by phone_number;
Show all records starting with the letters 'bob' AND the phone number '3444444'.

mysql> SELECT * FROM [table name] WHERE name like "Bob%" AND phone_number = '3444444';
Show all records starting with the letters 'bob' AND the phone number '3444444' limit to records 1 through 5.

mysql> SELECT * FROM [table name] WHERE name like "Bob%" AND phone_number = '3444444' limit 1,5;
Use a regular expression to find records. Use "REGEXP BINARY" to force case-sensitivity. This finds any record beginning with a.

mysql> SELECT * FROM [table name] WHERE rec RLIKE "^a";
Show unique records.

mysql> SELECT DISTINCT [column name] FROM [table name];
Show selected records sorted in an ascending (asc) or descending (desc).

mysql> SELECT [col1],[col2] FROM [table name] ORDER BY [col2] DESC;
Return number of rows.

mysql> SELECT COUNT(*) FROM [table name];
Sum column.

mysql> SELECT SUM(*) FROM [table name];
Join tables on common columns.

mysql> select lookup.illustrationid, lookup.personid,person.birthday from lookup left join person on lookup.personid=person.personid=statement to join birthday in person table with primary illustration id;
Creating a new user. Login as root. Switch to the MySQL db. Make the user. Update privs.

# mysql -u root -p
mysql> use mysql;
mysql> INSERT INTO user (Host,User,Password) VALUES('%','username',PASSWORD('password'));
mysql> flush privileges;
Change a users password from unix shell.

# [mysql dir]/bin/mysqladmin -u username -h hostname.blah.org -p password 'new-password'
Change a users password from MySQL prompt. Login as root. Set the password. Update privs.

# mysql -u root -p
mysql> SET PASSWORD FOR 'user'@'hostname' = PASSWORD('passwordhere');
mysql> flush privileges;
Recover a MySQL root password. Stop the MySQL server process. Start again with no grant tables. Login to MySQL as root. Set new password. Exit MySQL and restart MySQL server.

# /etc/init.d/mysql stop
# mysqld_safe --skip-grant-tables &
# mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("newrootpassword") where User='root';
mysql> flush privileges;
mysql> quit
# /etc/init.d/mysql stop
# /etc/init.d/mysql start
Set a root password if there is on root password.

# mysqladmin -u root password newpassword
Update a root password.

# mysqladmin -u root -p oldpassword newpassword
Allow the user "bob" to connect to the server from localhost using the password "passwd". Login as root. Switch to the MySQL db. Give privs. Update privs.

# mysql -u root -p
mysql> use mysql;
mysql> grant usage on *.* to bob@localhost identified by 'passwd';
mysql> flush privileges;
Give user privilages for a db. Login as root. Switch to the MySQL db. Grant privs. Update privs.

# mysql -u root -p
mysql> use mysql;
mysql> INSERT INTO db (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv) VALUES ('%','databasename','username','Y','Y','Y','Y','Y','N');
mysql> flush privileges; 

or 

mysql> grant all privileges on databasename.* to username@localhost;
mysql> flush privileges;
To update info already in a table.

mysql> UPDATE [table name] SET Select_priv = 'Y',Insert_priv = 'Y',Update_priv = 'Y' where [field name] = 'user';
Delete a row(s) from a table.

mysql> DELETE from [table name] where [field name] = 'whatever';
Update database permissions/privilages.

mysql> flush privileges;
Delete a column.

mysql> alter table [table name] drop column [column name];
Add a new column to db.

mysql> alter table [table name] add column [new column name] varchar (20);
Change column name.

mysql> alter table [table name] change [old column name] [new column name] varchar (50);
Make a unique column so you get no dupes.

mysql> alter table [table name] add unique ([column name]);
Make a column bigger.

mysql> alter table [table name] modify [column name] VARCHAR(3);
Delete unique from table.

mysql> alter table [table name] drop index [colmn name];
Load a CSV file into a table.

mysql> LOAD DATA INFILE '/tmp/filename.csv' replace INTO TABLE [table name] FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (field1,field2,field3);
Dump all databases for backup. Backup file is sql commands to recreate all db's.

# [mysql dir]/bin/mysqldump -u root -ppassword --opt >/tmp/alldatabases.sql
Dump one database for backup.

# [mysql dir]/bin/mysqldump -u username -ppassword --databases databasename >/tmp/databasename.sql
Dump a table from a database.

# [mysql dir]/bin/mysqldump -c -u username -ppassword databasename tablename > /tmp/databasename.tablename.sql
Restore database (or database table) from backup.

# [mysql dir]/bin/mysql -u username -ppassword databasename < /tmp/databasename.sql
Create Table Example 1.

mysql> CREATE TABLE [table name] (firstname VARCHAR(20), middleinitial VARCHAR(3), lastname VARCHAR(35),suffix VARCHAR(3),officeid VARCHAR(10),userid VARCHAR(15),username VARCHAR(8),email VARCHAR(35),phone VARCHAR(25), groups VARCHAR(15),datestamp DATE,timestamp time,pgpemail VARCHAR(255));
Create Table Example 2.

mysql> create table [table name] (personid int(50) not null auto_increment primary key,firstname varchar(35),middlename varchar(50),lastnamevarchar(50) default 'bato');
MYSQL Statements and clauses
ALTER DATABASE

ALTER TABLE

ALTER VIEW

ANALYZE TABLE

BACKUP TABLE

CACHE INDEX

CHANGE MASTER TO

CHECK TABLE

CHECKSUM TABLE

COMMIT

CREATE DATABASE

CREATE INDEX

CREATE TABLE

CREATE VIEW

DELETE

DESCRIBE

DO

DROP DATABASE

DROP INDEX

DROP TABLE

DROP USER

DROP VIEW

EXPLAIN

FLUSH

GRANT

HANDLER

INSERT

JOIN

KILL

LOAD DATA FROM MASTER

LOAD DATA INFILE

LOAD INDEX INTO CACHE

LOAD TABLE...FROM MASTER

LOCK TABLES

OPTIMIZE TABLE

PURGE MASTER LOGS

RENAME TABLE

REPAIR TABLE

REPLACE

RESET

RESET MASTER

RESET SLAVE

RESTORE TABLE

REVOKE

ROLLBACK

ROLLBACK TO SAVEPOINT

SAVEPOINT

SELECT

SET

SET PASSWORD

SET SQL_LOG_BIN

SET TRANSACTION

SHOW BINLOG EVENTS

SHOW CHARACTER SET

SHOW COLLATION

SHOW COLUMNS

SHOW CREATE DATABASE

SHOW CREATE TABLE

SHOW CREATE VIEW

SHOW DATABASES

SHOW ENGINES

SHOW ERRORS

SHOW GRANTS

SHOW INDEX

SHOW INNODB STATUS

SHOW LOGS

SHOW MASTER LOGS

SHOW MASTER STATUS

SHOW PRIVILEGES

SHOW PROCESSLIST

SHOW SLAVE HOSTS

SHOW SLAVE STATUS

SHOW STATUS

SHOW TABLE STATUS

SHOW TABLES

SHOW VARIABLES

SHOW WARNINGS

START SLAVE

START TRANSACTION

STOP SLAVE

TRUNCATE TABLE

UNION

UNLOCK TABLES

USE

String Functions
AES_DECRYPT

AES_ENCRYPT

ASCII

BIN

BINARY

BIT_LENGTH

CHAR

CHAR_LENGTH

CHARACTER_LENGTH

COMPRESS

CONCAT

CONCAT_WS

CONV

DECODE

DES_DECRYPT

DES_ENCRYPT

ELT

ENCODE

ENCRYPT

EXPORT_SET

FIELD

FIND_IN_SET

HEX

INET_ATON

INET_NTOA

INSERT

INSTR

LCASE

LEFT

LENGTH

LOAD_FILE

LOCATE

LOWER

LPAD

LTRIM

MAKE_SET

MATCH    AGAINST

MD5

MID

OCT

OCTET_LENGTH

OLD_PASSWORD

ORD

PASSWORD

POSITION

QUOTE

REPEAT

REPLACE

REVERSE

RIGHT

RPAD

RTRIM

SHA

SHA1

SOUNDEX

SPACE

STRCMP

SUBSTRING

SUBSTRING_INDEX

TRIM

UCASE

UNCOMPRESS

UNCOMPRESSED_LENGTH

UNHEX

UPPER

Date and Time Functions
ADDDATE

ADDTIME

CONVERT_TZ

CURDATE

CURRENT_DATE

CURRENT_TIME

CURRENT_TIMESTAMP

CURTIME

DATE

DATE_ADD

DATE_FORMAT

DATE_SUB

DATEDIFF

DAY

DAYNAME

DAYOFMONTH

DAYOFWEEK

DAYOFYEAR

EXTRACT

FROM_DAYS

FROM_UNIXTIME

GET_FORMAT

HOUR

LAST_DAY

LOCALTIME

LOCALTIMESTAMP

MAKEDATE

MAKETIME

MICROSECOND

MINUTE

MONTH

MONTHNAME

NOW

PERIOD_ADD

PERIOD_DIFF

QUARTER

SEC_TO_TIME

SECOND

STR_TO_DATE

SUBDATE

SUBTIME

SYSDATE

TIME

TIMEDIFF

TIMESTAMP

TIMESTAMPDIFF

TIMESTAMPADD

TIME_FORMAT

TIME_TO_SEC

TO_DAYS

UNIX_TIMESTAMP

UTC_DATE

UTC_TIME

UTC_TIMESTAMP

WEEK

WEEKDAY

WEEKOFYEAR

YEAR

YEARWEEK

Mathematical and Aggregate Functions
ABS

ACOS

ASIN

ATAN

ATAN2

AVG

BIT_AND

BIT_OR

BIT_XOR

CEIL

CEILING

COS

COT

COUNT

CRC32

DEGREES

EXP

FLOOR

FORMAT

GREATEST

GROUP_CONCAT

LEAST

LN

LOG

LOG2

LOG10

MAX

MIN

MOD

PI

POW

POWER

RADIANS

RAND

ROUND

SIGN

SIN

SQRT

STD

STDDEV

SUM

TAN

TRUNCATE

VARIANCE

Flow Control Functions
CASE

IF

IFNULL

NULLIF

Command-Line Utilities
comp_err

isamchk

make_binary_distribution

msql2mysql

my_print_defaults

myisamchk

myisamlog

myisampack

mysqlaccess

mysqladmin

mysqlbinlog

mysqlbug

mysqlcheck

mysqldump

mysqldumpslow

mysqlhotcopy

mysqlimport

mysqlshow

perror

Perl API - using functions and methods built into the Perl DBI with MySQL
available_drivers

begin_work

bind_col

bind_columns

bind_param

bind_param_array

bind_param_inout

can

clone

column_info

commit

connect

connect_cached

data_sources

disconnect

do

dump_results

err

errstr

execute

execute_array

execute_for_fetch

fetch

fetchall_arrayref

fetchall_hashref

fetchrow_array

fetchrow_arrayref

fetchrow_hashref

finish

foreign_key_info

func

get_info

installed_versions


last_insert_id

looks_like_number

neat

neat_list

parse_dsn

parse_trace_flag

parse_trace_flags

ping

prepare

prepare_cached

primary_key

primary_key_info

quote

quote_identifier

rollback

rows

selectall_arrayref

selectall_hashref

selectcol_arrayref

selectrow_array

selectrow_arrayref

selectrow_hashref

set_err

state

table_info

table_info_all

tables

trace

trace_msg

type_info

type_info_all

Attributes for Handles

PHP API - using functions built into PHP with MySQL
mysql_affected_rows

mysql_change_user

mysql_client_encoding

mysql_close

mysql_connect

mysql_create_db

mysql_data_seek

mysql_db_name

mysql_db_query

mysql_drop_db

mysql_errno

mysql_error

mysql_escape_string

mysql_fetch_array

mysql_fetch_assoc

mysql_fetch_field

mysql_fetch_lengths

mysql_fetch_object

mysql_fetch_row

mysql_field_flags

mysql_field_len

mysql_field_name

mysql_field_seek

mysql_field_table

mysql_field_type

mysql_free_result

mysql_get_client_info

mysql_get_host_info

mysql_get_proto_info

mysql_get_server_info

mysql_info

mysql_insert_id

mysql_list_dbs

mysql_list_fields

mysql_list_processes

mysql_list_tables

mysql_num_fields

mysql_num_rows

mysql_pconnect

mysql_ping

mysql_query

mysql_real_escape_string

mysql_result

mysql_select_db

mysql_stat

mysql_tablename

mysql_thread_id

mysql_unbuffered_query