What are the issues in using Full Text Indexing?

Reply 1)

When we deploy full-text indexing, we select an individual public folder or mailbox store to be indexed & then conduct full-text searches on the messages and attachments contained in the public folder or mailbox store. By default, the index contains the subject and body of a message, along with names of the sender and recipient and any names that appear in the Cc and Bcc fields. The index also includes text from the following types of attachments: .doc, .xls, .ppt, .html, .htm, .asp, .txt, and .eml (embedded Multipurpose Internet Mail Extensions (MIME) messages) files.

Issues in using full test indexing :

Binary attachments, such as pictures and sounds, are not indexed.

Search results are only as accurate as the last time the index was updated. Because the content of public folders or mailbox stores changes, the index must be updated to reflect the new content.

Reply 2)

Few more issues in using Full Text Indexing are listed below:

1) Index data generated for Full Text Indexing can eat upto 10 – 40% of the store size
2) Indexing is a CPU intensive process and CPU utilization can reach upto 90% during an index refresh

Sending emails using scripts

Here is the deal..

Yesterday I built my exchange machine .. fresh VM.

Joined it to a brand new domain. (Shantanu.local)
1.Created a 100 users using a script {Script} (Apptixuser1 to Apptixuser100)
2.Created mailboxes for all these 100 users using Dsa.msc (Selected all users , Right click -> Exchange task -> Create Mailboxes)

3.Now I want to grow my database size to few GB’s.

Step 3 is little complicated as I can’t find any way to send mails using scripts.

Please note that I am trying to send different emails.
When I send a email with attachment to all my users in domain – let’s say I have a attachment of 30MB and I send same mail to all the users in my domain in a single mail – my database size is growing only 30MB.
Can anyone please explain what’s going on in background.. ??

Please help me to increase my database size and also help me to understand better how database is growing..

Reply 1)

This is called ‘Single Instance Storage’, which works as follows:

1) 1 mail is sent to 5 mailboxes (or any number) with a 2 MB attachment
2) Exchange stores a single copy of the mail in the database and creates a link for this message in each mailbox. So a total of 6 links are created for a single message. 5 in Recipient’s Inbox and 1 in sender’s Sent Items folder
3) Due to this reason, the database size will grow only by 2 MB and not by 10 MB
4) The idea behind this was to save storage size in earlier versions of Exchange, because the Standard Edition database was restricted to 16 GB max size

There are few exceptions to this rule:
1) Single Instance Storage is maintained only within the same mailbox store. So in case the above 5 mailboxes are located on 5 different stores, a copy of the message will be stored on each store and the collective size of all 5 mailbox stores will grow by 10 MB (2 MB each)
2) Single Instance storage is broken if a message is modified. For example, if all 5 mailboxes are located on the same store, but 1 of the recipients opens the mail, and makes a small change and ‘Saves’ it again, it is stored as an additional copy, thereby increasing the size of the store by 2 MB to a total of 4 MB.
3) Single Instance storage is broken after an ExMerge. For example, if all 5 mailboxes are stored on the same store and someone Exmerges them out, they will consume 10 MB. Single Instance Storage will not be recreated after importing the exmerged data back into the store. The store will then have a size of 10 MB.
4) Lets say, 3 out of 5 messages are moved to a different store. In the new store also, a single copy of the message will be stored, with links to 3 moved mailboxes.

Hence one of the criteria for designing the structure for multiple stores is by grouping together people who need to send mails within themselves, for example, department or location.

Do let me know, if anyone has any queries.

What is Physical Address Extension (PAE) and its relevance with Exchange 2003?

Reply 1)

Microsoft has improved memory management for Exchange through techniques such as Dynamic Buffer Allocation within the Exchange Server 2003 Informaton store. However, Exchange developers still must depend on the underlying hardware and OS to effectively use memory. And since we don’t yet have a 64-bit version of Exchange, Exchange is limited to 4GB of address space.However, recent OS and hardware advances help Exchange make the most of 32-bit.

Physical Address Extension (PAE) is a hardware technology that lets windows applications on IA-32 servers address more than 4 GB of physical memory.PAE actually uses 36 bits to create additional addressable memory. PAE lets an OS memory manager use a three-level address-translation scheme to access memory above the 2 GB or 3 GB available on servers that don’t use PAE.

For more information refer the following link-
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/techref/26eccf33-2454-4222-841a-c6d5aa1fc54c.mspx. . . .

Reply 2)

The PAE switch is applied in the BOOT.ini file at the end of the OS line

It operates in 2 modes, /PAE (turns PAE on) and /NOPAE (turns PAE off).

PAE was disabled by default in Windows 2003 RTM, but was enabled by default with SP1.

Error -1018 in Exchange 2003

Why is -1018 such a dreaded error in Exchange 2003?
What are the possible situations when you see it?
How does it impact your operations?
How do you fix it?

Reply 1)

Why is -1018 such a dreaded error in Exchange 2003?

This error indicates Hard drive error and this causes dismounting the database(s). Even if you try to mount the database you will not be able to mount the database and you might get error c1041724.

You will the -1018 error during offline defrag, but it is only if you are using any different version of eseutil utility, like if your database is belongs to 2003 and you are using eseutil of Exchange 2000 then you will get -1018 error during offline defrag process.

What are the possible situations when you see it?

If there is a problem with Hard drive then you will find a event id 474 in your Application log which mentions in the description that there could be a problem with your Hard drive. You will see the event id as below:

Date: date
Source: ESE
Time: time
Category:
Type: Error
Event ID: 474
User: N/A
Computer: Servername
Description: Information Store (2240) The database page read from the file "E:\program files\exchsrvr\mdbdata\priv1.edb" at offset 204275712 (0x000000000c2d0000) for 4096 (0x00001000) bytes failed verification due to a page checksum mismatch. The expected checksum was 303571876 (0x121823a4) and the actual checksum was 303571940 (0x121823e4). The read operation will fail with error -1018 (0xfffffc06). If this condition persists then please restore the database from a previous backup.

If you see -1018 value it means your hard drive is in trouble and you will not be able to mount the store due to bad hard drive. You need to check since when you are getting this error.

In this case you have only option is replace the bad hard drive and restore the good backup with good log files. Reply the log files with the restored backup and mount the store. You might loose the data if you don't have recent good backup.

You might not be able to mount the database(s), if the database is mounted client might facing a problem of slow connectivity. Most of the clients will not be able to access their attachments, etc.

How does it impact your operations?

If the database(s) is dismounted nobody will be able to access their mails. Database dismounting is nothing but a business impact.

How do you fix it?

You need to replace the Bad Hard drive and restore the recent good backup with good log files, ensuring that you click to clear the Last Backup Set check box in ESM. after restore, run eseutil /mk on log files to check the consistency of the log files and then replay the log files with the database by running a command eseutil /cc /t and hit enter. Then mount the database.

Reply 2)

Apart from dismounting databases, which becomes the extreme case, how else can -1018 impact operation? In other words, what things can’t be done on an Exchange server which is reporting -1018?

Reply 3)

One of the key impact you will notice due to a -1018 error is the inability to conduct online backups. Following is the sequence of events that lead to a -1018 error during an online backup

1) Exchange EDB files (not stm) stores contents in 4 KB pages (8 KB for AD and Exchange 2007)
2) In order to maintain the integrity of the data stored in each page, the system computes a checksum and includes it with the data
3) An online backup operation works by reading each page of the edb file and after verifying the checksum, writes the data to tape (or other media)
4) In case, the checksum at the time of writing does not match at the time of reading during backup, the system will presume that the integrity of the data has been compromised.
5) It will then abort the backup process and write an error in the Application log with -1018

Domain Controller and Global Catalog Scenario

Scenario:

Lets say we have 100 DCs out of which 60 are GCs. We use a multi-site environment. How to know from a single console, which are the GCs in the domain?

Reply 1)

We can check the GC’s from AD sites and Services.

Reply 2)

This can be achieved by using the Replication Monitor tool (Replmon.exe). This is a tool available from Windows 2003 Support folders. Connect to any domain Controller using ReplMon, and right-click the server name. Choose Show Global Catalog Servers in Enterprise to display a list of all Global Catalog servers in the entire forest.

How to determine if there are duplicate SMTP addresses in Active Directory?

Reply 1)

If the SMTP address already exists within the organization, Active Directory Users and Computers informs you of this fact via the error message shown below:


Reply 2)

Simplest way to check will be type the address in outlook and hit ctrl+k. If the name get resolved the address is already in list. In case the name is hidden from Gal you can send a test mail with receipt to that address. If you receive the receipt the address is already existing.
Correct me if I am wrong.

Reply 3)

Alright, let me rephrase the question in the correct way.

Lets say we have a Domain Controller named, DC1 in India and another DC2 in US. Both these DCs are in their own AD sites and the replication is configured to happen in 3 hours.

An admin on DC1 is trying to create a user, say User1 with SMTP address of user1@domain.com and at the same time an admin in US is trying to create a user, User1 with SMTP address of user1@domain.com. What will happen when the 2 DCs synchronize?

Reply 4)

Logically Speaking … AD creates a conflicting (CNF:[GUID]) object if it finds the same object because of delay in replication. However as per the document

Duplicated SMTP Address
http://www.microsoft.com/technet/prodtechnol/exchange/guides/ExMgmtGuide/93733b19-b78c-4844-b263-04230026d9b6.mspx?mfr=true

Which says …
==============================================================================================================================================================================
When more than one mail-enabled object in Active Directory has the same SMTP address, the sender of an e-mail to any of those recipients will receive a non-delivery report (NDR) back with an NDR code of 5.1.4. (See Non-Delivery Reports.) The issue occurs because of the multimaster nature of Active Directory, when administrators create multiple mail-enabled objects on different domain controllers using the same SMTP address. If this occurs:
• Check to make sure no duplicated SMTP address exists in Active Directory.
• Consider using the LDAP Data Interchange Format Data Exchange (LDIFDE) tool to export the Active Directory database and search for duplicates.
==============================================================================================================================================================================

It seems .. it will create a Duplicate SMTP … but any one of them will receive an NDR.

Reply 5)

Exactly. Now, lets say we have lots of such duplicate SMTP addresses in our domain, resulting in loads of NDRs being generated. How can we find out which user names have got duplicate SMTP addresses? Any another method than using LDIFDE?

Reply 6)

I think we can also find using CSVDE. I will try to find command.
Correct me if I am wrong.

Reply 7)

You can use DUPSMTP.vbs downloadable from http://www.swinc.com/resources/ to achieve this as well.

Why should Exchange 2003 server not be installed on Windows 2000?

Reply 1)

Windows 2000 or with sp2 is not equal to Win 2003. To take full advantage and functionality of xchange 2003 you must run it on win 2003. Following is the list of feature of xchange 03 which are only supported on win 03 and not on win 2000.

@ Mount points overcome the 24-drive letter limitation of previous version of windows.
@ Volume shadow copy service for database backup
@ IPSec support for front and back end cluster
@ Cross-forest Kerberos authentication with MS Outlook 03
@ IIS 6 enhanced security and dedicated application mode
@ HTTP access from Outlook 2003
@ Real-time collaboration
@ Microsoft SharePoint Portal Server Web Part

List of function supported only when xchange 03 installed on win 03

@ Support 8-way PIII Xeon Processor
@ Support 8-way P4 XeonMP Processor (Hyper Threaded)
@ Up to 8-node Clustering
@ Mount Point Support.

For more information refer Book 70-284.

Reply 2)

Exchange 2003 requires IIS Version 6 where as in Windows 2000 has IIS Version 5. If you have Windows 2000 with SP3 or later you can install Exchange 2003 on that server.

Reply 3)

Both Exchange 2000 and Exchange 2003 rely heavily on Internet Information Services (IIS) in the Windows Server operating system for all Internet-based protocol services. For Windows Server 2003, IIS was extensively reengineered in accordance with industry best practices for increased system security. IIS in Windows Server 2003 now has two different modes: IIS 6.0 mode and IIS 5.0 compatibility mode. Improvements in IIS 6.0 include new fault tolerance that isolates applications in their own memory space and better protection against spam by disabling Internet Server Applications Programming Interfaces (ISAPI) by default. Additionally, IIS is not installed by default when running setup for Windows Server 2003; it is “locked down” to maximum security when selected for first installation on a server. These architectural design changes to IIS fundamentally change how Exchange and other applications utilize IIS in Windows Server 2003.

With approximately 350 code changes in Windows Server 2003 that affected Exchange, Microsoft determined that it would better benefit our customers to focus development efforts on Exchange 2003 to take full advantage of the new improvements in IIS 6.0, rather than on updating either version of Exchange to run on Windows Server 2003 in IIS 5.0 compatibility mode or Exchange 2000 to work with the new architecture in IIS 6.0. As a result, Exchange 2000 needs to be physically installed on a server running Windows 2000 with Service Pack (SP) 3 to coexist in a Windows Server 2003 environment. Exchange 2003 is fully supported on both Windows 2000 Server and Windows Server 2003 running IIS 6.0.

Reply 4)

Perfect guys, Nilesh mentioned below ‘HTTP access from Outlook 2003’. In case, someone did not understand what is this, it is RPC over HTTP which needs Exchange 2003 to be installed on Windows 2003.

Reply 5)

When I saw this mail from Nilesh, I got really impressed with the searching skills of Nilesh.
Keep up the spirit guys. This discussions are now getting really very informative.
I am enjoying it.

Only one question from my side..
What role IIS 6.0 plays here? Is really IIS6.0 is required to install Exchange 2003 ?
Or it just needs the IIS services running ? 5.0 / 6.0

Please help me to understand this better.
Thanking you all.

Reply 6)

What role IIS 6.0 plays here?
Answer: The following features are available in Exchange 2003 only with IIS 6
1) Exchange 2003 runs in IIS 6.0 as a ‘Worker Process’. What this means is, in case another process or a website on the same IIS server crashes, Exchange 2003 functionality is not impacted.
2) RPC over HTTP will NOT be available with IIS 5.0 since it is a feature of IIS 6.0

Is really IIS 6.0 is required to install Exchange 2003 ?
Answer: No. Exchange 2003 can run on IIS 5.0 as well, but will need .Net Framework and ASP.net to be installed. This is installed automatically during the Exchange 2003 setup on a Windows 2000 server. Refer http://technet.microsoft.com/en-us/library/bb124295.aspx

Or it just needs the IIS services running ? 5.0 / 6.0
Answer: If you are happy with losing out the points mentioned in Answer 1 above, Exchange 2003 can very well run on the HTTP, NNTP and ASP.net services provided by IIS 5.0. Refer to http://articles.techrepublic.com.com/5100-6345_11-5035139.html

Guys, please add more points if relevant. Thanks!

How do we send mail by SMTP Using Telnet Command?

Reply 1)

Step 1

Connect to the Internet in case you are a dial-up user. Open an MS-DOS prompt, and enter this command:
C:\WINDOWS>telnet mail.monitortools.com 25
This will open a Telnet window, and within a short time, you will be connected to the SMTP server, and the server says:
220 PROTAGONISTNT Mailmax version 4. 8. 3. 0 ESMTP Mail Server Ready
This varies, but you should definitely see the '220' part. It is an indication that the server is ready to service your request.
________________________________________

Step 2

Now the server expects you to identify yourself. If you are a dial-up user, you can enter the name of your computer (the one Windows asks you when you intall Windows) or anything else you want. If you have a domain-name, then you should enter the domain-name here. For eg: computer's name is dell01, so I say:
helo dell01
Note that it is 'helo' and not 'hello'. The commands are not case-sensitive, so you can also say HeLo or HELO or hELo. The server replies:
250 HELO 217.120.215.201, How you can I help?
This is like a shake-hand. You tell the server your name, and it says its name.
________________________________________

Step 3

Next give the server your e-mail address. Note that most SMTP servers require that your e-mail address belong to the same domain as the server. For example, if you send mail from Yahoo! SMTP server, you should have a Yahoo! address. You cannot use it if you give it a Hotmail address. Let me give the SMTP server some e-mail address:
mail from: webmaster@monitortools.com
'mail from:' is a SMTP command. Note that there is a space between 'mail' and 'from', followed by a colon (:). The server says:
250 Ok
________________________________________

Step 4

Tell the server who you want to send the e-mail to. Let me send a mail to info@activexperts:
rcpt to: info@activexperts.com
There are no restrictions here. You can enter any e-mail address. If there is some problem with the recipient-address, your mail will bounce, but for now, the server doesn't complain. It will say:
250 Ok
________________________________________

Step 5

You have told the server your e-mail address, and the recipient's e-mail address, so now you can go ahead and type the e-mail. You have to do that with the data command:
data
The server asks you to go ahead with your e-mail:
354 End data with .
Don't worry with the thing. It'll be explained later.
________________________________________

Step 6

Now type in your e-mail, like this:
This is a test e-mail.
Remember to type it all right. Backspace key doesn't work in Windows
Telnet, though it does in Linux. If you make a mistake, try pressing
CTRL-h. If it works, well and good.
When you finish your e-mail, press [ENTER], then a '.', and again an [ENTER]. This tells the server that you have finished the e-mail, and it can send it. It will say:
250 Ok: queued as 6AB5150038
Your mail was sent!
________________________________________

Step 7

Now you can either send another mail, or disconnect from the server. If you want to send another mail, you should repeat the 'rcpt to:' and 'data' commands. There is no need for 'helo' and 'mail from:', because the server already knows who you are. If you want to disconnect, just say 'quit':
quit
The server will reply:
221 Bye
and you will lose connection with the server

Reply 2)

Introduction

We shall again be using Telnet to talk to our remote server here, like POP. The principle behind sending an email is simple - your local computer connects to the remote mail server, talks to it using SMTP - "Simple Mail Transfer Protocol". When the mail is sent, the session is over and the remote server closes the connection.

When you use an email client like Outlook or Eudora, the mail client does all this for you. It automates the process of talking to your mail server to send and receive emails. But what if you don't have, or don't want to use, a mail client? We can use Telnet!

First choose "Run" in your Start menu and type in Telnet. Telnet is an application that allows us to communicate with remote computers. In this example, we shall be communicating with Yahoo's SMTP mail server. Choose "Remote System" from the "Connect" Menu. This will give you a box, with 3 input boxes. Type in the host name - the address of the mail server. For my Yahoo, the SMTP mail server is at smtp.mail.yahoo.com.

Now about the port: The port is a sort of a "gateway" to a computer. On the internet, each protocol, by convention has one or two port numbers assigned for itself. The HTTP connection is usually done using ports 80 and 8080; while POP transactions are done using port 110. For SMTP, port 25 is used. So type in 25 for the port.

Conversation with the Server

Now Click on connect. Once you're connected to the mail server, the mail server will respond with something like this:

220 smtp017.mail.yahoo.com ready.
Now we need to introduce ourselves to the computer and specify the sender's address. Technically, it is possible to use any SMTP server to send a mail with any server's name as the sender. This is called "Message Relaying". Since almost all servers have this feature turned off, we will simply type in the name of the SMTP server itself. [Note that you will not be able to see what you type.]

HELO smtp.mail.yahoo.com

The server will respond with:

250 Hello smtp.mail.yahoo.com, pleased to meet you.

Now we specify the sender:

MAIL From:
The server replies:

250 ... OK

There are a few observations to be made here - note that you can specify any sender here. So if you wanted to cheat the server and send bogus mail, the SMTP will not stop you - it has no security provisions. To add security, they combine the SMTP with POP authentication. So, you will have to login using the POP protocol once before using SMTP. [See Dec 2001 issue for POP mail]
Also note that whenever the server sends a message, there's a 3 digit code along with it.

For example, when it sends 250, it means that the Transaction's okay. If it's 220, it means Service Ready. If it's 500, it means there's been a syntax error in the command that you sent, and so on. There are lots of these codes, each having a specific meaning.

This is a very useful thing, as mail program using the protocol will not need to read any of the English text - they will simply read the code to understand what the response is.

Now we type in the recipient and the data, and then quit:

RCPT To:

250 ... Recipient ok

DATA

354 Enter mail, end with "." on a line by itself
From: anaplexian@yahoo.com
To: anaplexian@yahoo.com
Subject: Hi there!

This is a test message!
.

250 Mail accepted

QUIT

221 smtp.mail.yahoo.com delivering mail
[connection closed]

Take a look at the format of the email - it had a bunch of details like From, To, and Subject listed, and then I left a line and then started my email. This is because a normal email comprises of minimum two parts, the header and the body, which are separated by a blank line.

The moment you send this email and close transaction using QUIT, the mail server will send the mail off to its destination.

So now we can send email using SMTP, (and receive using POP) all without the use of a mail client or a web browser. Note that the commands we did are only a part of the whole list - there's a lot more you can do with SMTP and POP.

So next time you want to check your mail, do it the cool way - use Telnet!!

SMTP Cheat Sheet

List of Basic SMTP Commands:
HELO: identifies client

MAIL: identifies the sender of the message.

RCPT: identifies the recipient. More than one RCPT command can be issued if there
are multiple recipients.

DATA: To type in the message

QUIT: terminates conversation and closes connection.

What is RAID and what are its different types?

Reply 1)

RAID: Redundant Array of Independent Disks

The abbreviation for the RAID is Redundant Array of Independent Disks. This RAID by definition stands for the subsystem for the disks. The expectation for using this is the increment in the performance and the value added services in the reliability. The major purpose of the system is to provide the fault tolerance subsystem which can provide efficiency and reliability to the overall performance of the system. The RAID is also used as a server for the reasons mentioned above. The RAID in the earlier history is also implemented by the software to enable the present abilities.

Since the purpose of RAID is for fault tolerant systems hence the design is suited for that purpose. The RAID technology is actually a set of standards. These standards are required to be followed for developing a fault tolerant storage system. The performance also matters a lot here. Hence in the mentioned above paragraph it is said that RAID has been implemented by only the software. The set of standards should be kept in mind before implementing the RAID. This is done using at least two ordinary hard disks and a RAID controller.

The RAID has its origin starting from the year of 1980. At that time it was referred as the Redundant Array of Inexpensive Disks. This was in comparison with the storage system available at that time. The storage devices where quite expensive those days; so the implementation of a secure RAID drives was an important enhancement in the field of storage systems. Presently the prices of the memory whether it is the secondary memory like the hard disk, floppy drive, compact disk or any other storage media as well as the primary memory storage like the RAM etc, are all decreasing day by day. Hence by these statistics the RAID Advisory board modified the parameters from inexpensive to the independent.

The concept of mirroring and parity is also available in the RAID drives. In fact the property of fault tolerance is achieved by the process of mirroring and fault tolerance. The achievement is quite necessary for the purpose of providing a fault tolerant system.

The RAID system may have an altogether different drive for the sole purpose of replacing the drive that is failed or might have crashed. The RAID is drive that is replaced and is in spare is called as the hot spare. The hot drive is used in the case of an emergency where in the drive is the spare part that is used to fill in the gap provided by the crashed systems drive. Such a drive must always be ready and waiting. The physical state of such a drive is of quite importance where in the drive must be made available for the purpose of providing back up to the system. The replacement should be carried out immediately. So after the replacement is carried out now the entire system must be made aware of the fact that the hot spare drive is in use. And also the provision should be made for filling up the gap made by the hot spare drive. This is necessary if the other drives also fail and the condition is also possible even if the hot spare drive itself fails. But the RAID continues to dominate the technology that is used for the implantation of secure systems.


The different types of RAID levels are RAID 0, RAID 1, RAID 3, RAID 5 & RAID 10 levels.

RAID 0, STRIPING:
In this system, the data which is to be written across the drivers are split up in blocks of array.
RAID 0 will offers a superior Input Output performance and the performance can be increased further by using multiple controllers. The advantage of using RAID O is that it offers great performance such as read and writes operations. The Disadvantage of RAID 0 is not fault tolerant.

For Example: If at all the data in one of the disk is lost then all the data in the RAID 0 array will be lost. RAID 0 is designed for non critical storage of data where read and write are at a high speed. For example, it can be used in the Photoshop image retouching station.

RAID 1, MIRRORING:
In RAID 1, the data is stored twice on the data disk and on a mirror disk. If one of the disks fails, the controller uses the data drive or the mirror drive for data recovery. The advantages of using RAID 1 are excellent read speed and a write speed which is very high comparable to that compared to a single disk. If one of the disks fails, data is copied to the replacement disk. RAID 1 is a very simple technology compared to RAID O. The disadvantages of RAID 1 are that the storage capacity is half of the total disk capacity which is present in the system because all data get written twice. RAID 1 is ideally suited for mission critical storage. It is also suitable for small servers.

RAID 3:
In RAID 3 systems, the data blocks are divided into and are written in parallel on two or more drives. The additional drive which is used to stores parity information. Since parity is used in RAID 3 stripe set can handle a single disk failure without losing data. The advantages of RAID 3 are to provide high throughput for large data transfers. The disadvantage of RAID 3 is complex and performance is slower for small Input Output operations.

RAID 5:
RAID 5 is the most common used RAID level.
It is somewhat similar to RAID-3 in which data is transferred to disks by independent read and write operations. RAID 5 arrays can withstand a single disk failure as in RAID 3, without losing data. Extra cache memory can be provided in order to improve the write performance. The advantage of RAID 5 is it reads data transactions are very fast. The disadvantage of RAID 5 is disk failures and this is complex technology.

RAID 10, a mix of RAID 0 and RAID 1:
RAID 10 uses the advantages of RAID 0 and RAID 1 in a single system. Its added advantage helps in proving good security by mirroring all data on a secondary set of disks. The RAID 2, 4, 6 or 7 levels do exist in prepress environments. The advantages of RAID 10 are read data transactions are very fast & it is a very simple technology. The disadvantages of RAID 10 are that is its performance is slower for large transfers.

Reply 2)

RAID stands for Redundant Array of Independent (or Inexpensive) Disks,

There are number of different RAID levels:
Level 0:
Level 0 is a 'striped' disk array without fault tolerance. It provides data striping (spreading out blocks of each file across multiple disk drives) but no redundancy.
Level 1:
Level 1 does 'mirroring' and 'duplexing'. It provides disk mirroring

Level 2:
Level 2 does 'error-correcting coding'

Level 3:
Level 3 is 'bit-interleaved parity'. It provides byte-level striping with a dedicated parity disk

Level 4:
Level 4 is 'dedicated parity drive'. It is a commonly used implementation of RAID

Level 5:
Level 5 is 'block interleaved distributed parity'. It provides data striping at the byte level and also stripe error correction information.

Reply 3)

The distribution of data across multiple drives can be managed either by dedicated hardware or by software. Additionally, there are hybrid RAIDs that are partially software and hardware-based solutions.

Software RAID

Software implementations are provided by most operating systems. A software layer sits above the (generally block based) disk device drivers and provides an abstraction layer between the logical drives (RAID arrays) and physical drives. Software RAID is typically limited to RAID 0 (striping across multiple drives for increased space and performance), RAID 1 (mirroring two drives) and RAID 5 (data striping with parity).

In a multi-threaded operating system (such as Linux, FreeBSD, Mac OS X, Windows NT/2000/XP/Vista and Novell NetWare) the operating system can perform overlapped I/O, allowing multiple read or write requests to be initiated without waiting for completion on each request. This capability makes RAID 0/1 possible in an operating system. However, most operating systems do not support RAID 0/1 striping or mirroring with parity, due to the substantial processing demands of calculating parity].

Software implementations require some very small amount of processing time, which is provided by the main CPU in the host system. Since SCSI, PATA, and SATA drives all support asynchronous read/write, any multi-threaded operating system can support non-parity RAID on multiple hard drives with only a one percent increase in CPU overhead[ .

Software implementations can exceed the performance levels of hardware-based RAID due to the high-performance of modern CPUs]. Since the software must run on a host server attached to storage, the processor (as mentioned above) on that host must dedicate processing time to run the RAID software. Like hardware-based RAID, if the server experiences a hardware failure, the attached storage could be inaccessible for a period.

Software implementations can allow RAID arrays to be created from partitions rather than entire physical drives.

Hardware RAID

A hardware implementation of RAID requires at a minimum a special-purpose RAID controller. On a desktop system, this may be a PCI expansion card, or might be a capability built in to the motherboard. In industrial applications the controller and drives are provided as a standalone enclosure. The drives may be IDE/ATA, SATA, SCSI, SSA, Fibre Channel, or any combination thereof. The using system can be directly attached to the controller or, more commonly, connected via a SAN. The controller hardware handles the management of the drives, and performs any parity calculations required by the chosen RAID level.

Most hardware implementations provide a non-volatile read/write cache which, depending on the I/O workload, will improve performance. Cached RAID controllers are most commonly used in industrial applications.

Hardware implementations provide guaranteed performance, add no overhead to the local CPU complex and can support many operating systems, as the controller simply presents a logical disk to the operating system.

Hardware implementations also typically support hot swapping, allowing failed drives to be replaced while the system is running.

Hybrid RAID

Hybrid RAID implementations have become very popular with the introduction of inexpensive RAID controllers, implemented using a standard disk controller and then implementing the RAID in the controllers BIOS extension (for early boot-up/real mode operation) and the operating system driver (for after the system switches to protected mode). Since these controllers actually do all calculations typically proprietary to a given RAID controller manufacturer and typically cannot span multiple controllers. The only advantages over software RAID are that the BIOS can boot from them, and the tighter integration with the device driver may offer better error handling.

Both hardware and software implementations may support the use of hot spare drives, a pre-installed drive which is used to immediately (and almost always automatically) replace a drive that has failed. This reduces the mean time to repair period during which a second drive failure in the same RAID redundancy group can result in loss of data. It also prevents data loss when multiple drives fail in a short period, as is common when all drives in an array have undergone very similar use patterns, and experience wear-out failures

Reply 4)

Great posts everyone. Few queries that come to mind:

1) What is parity?
2) What are the possible ways of connecting a RAID system to the server?
3) Is there a minimum and maximum “number of disk” limit?
4) What is the difference between Disk Mirroring and Disk Duplexing?

Reply 5)

To gain performance and/or additional redundancy the Standard RAID levels( level 0 to level 5 ) can be combined to create hybrid or Nested RAID levels. Many storage controllers allow RAID levels to be nested. That is, one RAID can use another as its basic element, instead of using physical drives

For example, RAID 10 (or RAID 1+0) consists of multiple level 1 arrays stored on physical drives with a level 0 array on top, striped over the level 1 arrays. In the case of RAID 0+1, it is most often called RAID 0+1

Common nested RAID levels
RAID 0+1: Striped Set + Mirrored Set (4 disk minimum; Even number of disks) provides fault tolerance and improved performance but increases complexity. The key difference from RAID 1+0 is that RAID 0+1 creates a second striped set to mirror a primary striped set. The array continues to operate with one or more drives failed in the same mirror set, but if two or more drives fail on different sides of the mirroring, the data on the RAID system is lost.
RAID 1+0: Mirrored Set + Striped Set (4 disk minimum; Even number of disks) provides fault tolerance and improved performance but increases complexity. The key difference from RAID 0+1 is that RAID 1+0 creates a striped set from a series of mirrored drives. The array can sustain multiple drive losses as long as no two drives lost comprise a single pair of one mirror.
RAID 5+0: A stripe across distributed parity RAID systems
RAID 5+1: A mirror striped set with distributed parity

Also we can refer to following link for more info on nested array:
http://en.wikipedia.org/wiki/Nested_RAID_levels

Reply 6)

If the storage box is external, it can also have a Fibre interface. Also other option is the create virtual LUNs in SANs and then use RAID.

Reply 7)

1) What is parity?

Parity — Redundant information that is associated with a block of information and used to Rebuild a disk that has failed.

- RAID 5 arrays map data and parity intermittently across a set of disks. Within each stripe, the data on one disk is parity data and the data on the other disks are normal data. Therefore, RAID 5 arrays require at least three disks to allow for this Parity information. When a disk fails, the Array Manager software uses the parity Information in those stripes in conjunction with the data on the other disks to re-create the data on the failed disk.


2) What are the possible ways of connecting a RAID system to the server?

Possible ways of connecting RAID system is SCSI.
SCSI — Acronym for small computer system interface, which is a type of interface between a system and devices such as hard drives, diskette drives, CD drives, printers, scanners, and other peripherals.

3) Is there a minimum and maximum “number of disk” limit?


4) What is the difference between Disk Mirroring and Disk Duplexing?

Disk duplexing is a variation of disk mirroring in which each of multiple storage disks has its own SCSI controller. Disk mirroring (also known as RAID-1) is the practice of duplicating data in separate volumes on two hard disks to make storage more fault-tolerant. Mirroring provides data protection in the case of disk failure, because data is constantly updated to both disks. However, since the separate disks rely upon a common controller, access to both copies of data is threatened if the controller fails. Disk duplexing overcomes this problem; the use of redundant controllers enables continued data access as long as one of the controllers continues to function.

This failover method helps to ensure that data access will continue transparently to the user and allows technicians to take the server down to replace the defective controller at a more opportune time, instead of at the moment of failure. The ability to choose when the server comes down can be very advantageous, because -- in accordance with Murphy's Laws of Information Technology (Law of Inconvenient Malfunction) -- a device is likely to fail at the least opportune possible moment. Nevertheless, some experts advocate other systems (such as higher level RAID configurations) that don't require taking the server down to replace defective hardware.

Another benefit of disk duplexing is increased throughput. Using a technique known as a split seek, whichever disk can deliver the requested data more quickly responds. Multiple requests may also be split between the disks for simultaneous processing.

Reply 8)

I think already everyone is aware about the RAID.
Though I would like to add some images which will be more helpful in understanding of RAID Functionality.

RAID 0


RAID 1


Below we are looking at the RAID 1+0 i.e. RAID 10. Please find the exact description for this diagram in anjum’s Email.


RAID 4


RAID 5

RAID 5 divides the data and creates parity information similar to RAID 4, unlike RAID 4 the parity data is written separately across multiple disks.


RAID 6

RAID 6 deploys two parity records to different disk drives (double parity) enabling two simultaneous disk drive failures in the same RAID group to be recovered.

Why is ICMP considered dangerous?

Reply 1)

ICMP is the protocol used by the ping command. Why is it considered dangerous and recommended to be banned by network administrators?

Reply 2)

ICMP Overview –
The Internet Control Message protocol was originally created to allow the reporting of a small set of error conditions. However it is used to implement a wide range of error-reporting, feedback, and testing capabilities. It is a companion protocol added to IP to overcome the flaws in IP like connectionless, unreliable, and unacknowledged. ICMP provides support to IP that allow different types of communication to occur between IP devices. These messages use a common general format and are encapsulated in IP datagrams for transmission. The key concept is in TCP/IP, diagnostic, test, and error-reporting functions at the internetwork layer are performed by the ICMP. The original version, now called ICMPv4, is used with IPv4, and the newer ICMPv6 is used with IPv6. I found a table but have no idea what it says can anyone explain to me. I will research bit more on this.

Table 31-1:


Reply 3)

It’s dangerous because so called term “ping of death ” … which means thousands of ping simultaneously …. may be because of a virus …. can affect .

Reply 4)

Yeps, the primary reason to block ICMP is to avoid any sort of compromise in terms of security of the network. Unblocking ICMP makes life of a hacker very easy to intrude the environment. Secondly, network traffic also increases as ICMP echo is sent to each host on the network in order to identify the open port.

If one does not want block ICMP then NAT (Network Address Translation) might be used which allows only specific IP addresses to connect to the target machine. For instance, if Group policies are not applying due to slow link detection then, ICMP is required to check if any packets are being fragmented.

For more information, refer http://www.microsoft.com/technet/technetmag/issues/2005/01/AnatomyofaHack/

Reply 5)

ICMP can be dangerous because Hackers can use it to map & attack networks. So it needs to be restricted.

Reply 6)

IPSec (IP Security policy) can also be used to block certain ports and protocols. They can allow or deny the incoming/outgoing traffic to target machine.

Reply 7)

Very correct. Following are some reasons which add up to already discussed points in the forum and some are new

1) ICMP can be used to launch Denial of Service attacks (DoS). Mihir touched upon this earlier. This works by overloading a server with a particular (ping) request, in such a way, that the server cannot process anything. Thereby, it is not able to serve its primary goal, of maybe a web server or an Exchange Frond End server. One example of this is Smurf. Someone, please provide some details on this if possible.
2) Using ICMP, hackers can get too much information about a system. This was designed to help troubleshoot network issues, but using it in the wrong way can mean misusing the information. Hackers use the information gained from ICMP to impersonate other systems. Example, SPAM and virus mails are generally never distributed from the spammers or virus creators machines.
3) To make matters work, ICMP was not designed to use authentication. Hence it is all the more vulnerable
4) ICMP also provides OS Fingerprinting. This means that using ICMP, it is possible to know what OS is installed on the target machine. So, the hacker knows that a particular machine has Windows 2003 and using the port scanning feature of ICMP, he can also know which services are running over which port. If he has an exploit ready for this, then God save your server. Hence it also becomes important to install the critical patches released by Microsoft, since they patch the vulnerabilities which they know exist and are known to hackers.