Check technical reports 3437 and 3838 for sizing raid group for best performance
options raid.timeout 0 - this disables the automatic shutdown which occurs after 24 hrs once two drives have failed in a raid group and no spares available. RUN disk zero spares periodically to zero drives ahead of time to reduce time taken for spares and creating aggregates Run on each controller: options lun.use_partner.cc.warn_limit 300 options lun.use_partner.cc.bytes 2457600 https://kb.netapp.com/support/index?page=content&id=3010111&actp=LIST_POPULAR options nfs.export.auto-update off - disables the default feature which automatically exports every new LUN as an NFS exportoptions raid.background_disk_fw_update.enable - allows the automatic update of drive firmware in the background without downtime, a drive is taken out of a raid group one at time and updated then re-added.
I'd make a note that while FC shelf firmware upgrades will be non-disruptive, a SATA shelf will suspend IO to that shelf for ~30 seconds. Thus downtime for the services running from aggregates hosted on that shelf/loop may be required.
Just copy that 723_setup_e.exe file right to /etc/software, and use the "software install" command to expand it on the Filer. It's much quicker than using CIFS to copy the files as they are expanded.
And of course, always make sure you take a Snapshot of each volume, and make sure you send an Autosupport message before and after.
"options autosupport.doit pre-upgrade"
That'll make sure that NetApp Global Services has something to work with in case something does go wrong.
Better way to edit /etc/rc file telnet to the filer ----> issue rdfile /etc/rc, copy the contents to a text editor on your administration machine, include the entries for VLAN -----> issue wrfile /etc/rc on the telnet window -----> copy the contents from your text editor and paste on the telnet window -------> press Enter and CTRL + C. This will save the changes made to /etc/rc fileissue rdfile /etc/rc and check for changes made
WWNN - world wide node name WWPN - world wide port name
System Manager - mgmt clientFAS - fabric attached storage WAFL - write anywhere file layout Doing snapshots on luns different to doing snapshots on volumes Install Operations Manager/Protection manager to manager all filers and to get performance info etc. requires sql, check disk usage to know if to go vm or not successor to pam2 is flash cachetr3461 - vseries best practices guide
The below has been replaced with the new recommendation which is to autogrow the volume and auto delete old snapshots: Fractional Reserve should be renamed “Space to Protect the LUNs from going offline when writing changes after a snapshot has been taken”Fractional reserve and snapshots: volume = 2 x requested lun size + 20% for snap reserve The WAFL file system that Data OnTap uses actually writes a new block for any change. It will not modify the existing blocks. So for example you have a 100gb database in a 110gb lun and you re-index it, every change it makes results in a new block being created. That means you need 100gb of space set aside since the filesystem doesnt modify any blocks. With that said, you would need 220gb of space in the volume holding the 110gb lun which has the 100gb database in it just to allow you to re-index a database. (lun size + 100% of space for the new blocks to be created). If you want to take snapshots you need to add space beyond that. You should only reduce the fractional reserve setting is you are ABSOLUTELY sure your changes will never exceed the amount you specify. In other words if you expect you will never change more than 50% of the blocks in a volume you could set the reserve to 50%. Most database applications (such as sql and exchange) require or recommend that you leave the fractional reserve at 100% (to account for upgrades that may re-index the databases, defrags, etc
SAS Disk Shelf: DS4243 - the first 4 is the U size, the 24 is the # of disks, the 3 is the SAS speed (3Gb/s) SSD disks, good for random IOPs, read and write - data ONTAP 8.0.1 or later requiredSSD requests dont goto cache or flash cache
Flash cache - good fit when data is random read intensive, ssd drives better for both read and write IOPs Cache hits are good, misses are bad - data access from cache is much faster obviously Writes dont really hit flash cache, WAFL optimizes the writes Ontap 7.3 to version gx or cluster mode the data is wiped and converted Cluster mode allows up to 24 nodes - 12 pairs of controllers Write consistent (quiesced) vs crash consistent (not quiesced) - quiescing of data before snapshot is taken Baud settings - 9600,8-n-1,noneRemote LAN manager (RLM) or service processor
Controller Ports
e0M - management network e0P - private mgmt network Each will have their own IP
CLI has two modes:
1) Administrative priv set or priv set admin Console shows: system>
2) Advanced priv set advanced Console shows system*>
When you create an aggregrate the system will automatically select drives that are the same type, speed, checksum, and capacity.
<-----------------------Physical------------------------------------><.Logical---------------------------> Hard drive->RAID Groups->Plexes->Aggregrates->Volumes->Luns Or Qtrees
A Plex: when used with syncmirror provides mirror capabilities Contains one or more raid groups.
A Raid group provides data protection, contains 2 or more disks
Mirroring is done at the aggregrate level - both aggregrates must have the same # and type of disks
If you lose two disks in a raid dp raid group, and you have no hot spare, the system goes into degraded mode and will shutdown in 24 hrs. You can change that from command line using options raid.timeout
The 1st three drives per controller is for the ONTAP OS
32bit vs 64bit Aggregrate - 32bit appears to have better performance, 64bit allows you to exceed the 16TB aggregrate limit
11 PERFORMANCE This section is an overview of the performance characteristics of 64-bit aggregates. It also offers recommendations on the generic workloads that have better performance and those that will be affected by 64-bit aggregates. 64-bit aggregates enable you to add more data disks to an aggregate than do 32-bit aggregates. When you create a FlexVol volume inside a 64-bit aggregate, the FlexVol volume is striped across all the disks in the aggregate. Therefore the FlexVol volume has the capability to use more spindles. This means that for workloads and storage system configurations that were constrained by the number of available disk drive spindles, 64-bit aggregates can provide better performance if you can add enough drives such that the workload or storage system is no longer disk spindle constrained in delivering performance. 14 A thorough introduction to 64 Bit Aggregates 15 A thorough introduction to 64 Bit Aggregates 64-bit aggregates have a larger address space and need more memory for their metadata, compared to 32-bit aggregates. This extra memory usage reduces the amount of memory available for user data. Therefore, for workloads that are memory intensive, you may experience a slight performance impact when running the workload from a FlexVol volume contained in a 64-bit aggregate compared to running the workload from a volume contained in a 32-bit aggregate. Workloads that are highly random in nature typically access more metadata over a given period of time compared to sequential workloads. Random read workloads with a very large active data set size may experience a performance impact when run on a FlexVol volume in a 64-bit aggregate, compared to when run on a FlexVol volume in a 32-bit aggregate. This is because the data set size combined with the increased metadata size can increase memory pressure on the storage system and result in an increased amount of on disk I/O. In such scenarios, if you want to run the random workload from a volume contained in a 64-bit aggregate, using PAM improves the performance delivered by the storage system and helps alleviate any performance impact seen with 64-bit aggregates. Note that just having a 64-bit aggregate on the storage system does not result in any sort of performance degradation. The effects on performance, if any, are seen when data in any FlexVol volume in the 64-bit aggregate starts to be accessed. The overall performance of a storage system depends on a variety of factors, such as the configuration of the system, the workload, and activities on the storage system itself. For an analysis of your specific storage system performance, or to figure out whether a 64-bit aggregate is the right choice for your workload, contact your NetApp system engineer.
Volumes represent logical volumes
ONTAP allows up to 500 volumes
Following types supported: -Flex vol -traditional volumes (deprecated)
Flex volumes allow you to create very small or large (as big as aggregate) volumes. You can also expand and shrink volumes online (without disruption)
5.2 RAID GROUPS RAID group configuration can greatly affect a storage system’s resiliency. NetApp highly recommends using RAID-DP for all storage configurations, because it offers the best resiliency features and enables nondisruptive background firmware updates for drives. The best practices and points discussed in this section assume the use of RAID-DP. It is tempting to always create the largest RAID groups in an aggregate to minimize parity tax and maximize performance, but the results of doing so are: Larger failure domains: The more drives you have in a RAID group, the more likely it is that one or more of those drives will fail in the course of the operational lifetime of the storage system. Drive reliability is a primary factor in attempting to understand the risk of encountering multiple drive failures (MDFs) within a single RAID group. Ultimately any calculation is a guess because there is no guarantee that drives will fail all at the same time, in the same RAID group, or fail at all (within the five-year warranty period). Increased drive reconstruction times: The more data drives that are present in the RAID group, the greater the calculation overhead for reconstructing data from parity. Each data drive contributes a data point that needs to be considered in the parity calculations. The more data points, the larger the parity calculation is, and, as a result, the reconstruction times increase. In RAID group sizes from 12 to 20, data suggests that this increase is as little as 6%.
QTrees
A logically defined file system within a volume A special subdirectory at the root of a volume Viewed as a directory by clients
Qtrees allow you to further partition data within a volume Establish unique quotas for the restricting space Perform backup and recover with snapvault software Perform logical mirroring with snapmirror software 4,995 max qtrees per volume
The qtree command creates qtrees and specifies attributes for qtrees. A qtree is similar in concept to a partition. It creates a subset of a volume to which a quota can be applied to limit its size. As a special case, a qtree can be the entire volume. A qtree is more flexible than a partition because you can change the size of a qtree at any time
Writes are made to RAM and NVRAM, then disks (NOT flash cache)
Reads are made from RAM and disks or flash cache
Changing the size of RAID groupsMaximum and default RAID group sizes You can change the size of RAID groups that will be created on an aggregate or a traditional volume. Maximum and default RAID group sizes vary according to the storage system and level of RAID group protection provided. The default RAID group sizes are generally recommended. Maximum and default RAID-DP group sizes and defaultsThe following table lists the minimum, maximum, and default RAID-DP group sizes supported on NetApp storage systems. Storage system Minimum group size Maximum group size Default group size All storage systems (with SATA disks) 3 16 14 All storage systems (with FC disks) 3 28 16
Maximum and default RAID4 group sizes and defaultsThe following table lists the minimum, maximum, and default RAID4 group sizes supported on NetApp storage systems. Storage system Minimum group size Maximum group size Default group size FAS250 2 14 7 All other storage systems (with SATA disks) 2 7 7 All other storage systems (with FC disks) 2 14 8
Note If, as a result of a software upgrade from an older version of Data ONTAP, traditional volumes exist that contain RAID4 groups larger than the maximum group size for the storage system, convert the traditional volumes in question to RAID-DP as soon as possible. Changing the maximum size of RAID groups The aggr option raidsize option specifies the maximum RAID group size that can be reached by adding disks to an aggregate. You can change the value of this option for an aggregate, as long as you stay within the minimum and maximum values specified inMaximum and default RAID group sizes. The following list outlines some facts about changing the raidsize aggregate option:You can increase the raidsize option to allow more disks to be added to the most recently created RAID group.The new raidsize setting also applies to subsequently created RAID groups in an aggregate. Either increasing or decreasing raidsize settings will apply to future RAID groups.You cannot decrease the size of already created RAID groups.Existing RAID groups remain the same size they were before the raidsize setting was changed. Changing the raidsize settingTo change the raidsize setting for an existing aggregate, complete the following step. Step Action 1 Enter the following command: aggr options aggr_name raidsize size aggr_name is the aggregate whose raidsize setting you are changing. size is the number of disks you want in the most recently created and all future RAID groups in this aggregate. ExampleThe following command changes the raidsize setting of the aggregate yeraggr to 16 disks: aggr options yeraggr raidsize 16
NFS
Once you setup NFS you have to ‘export’ the resource to make it available to clients
-Edit the /etc/exports file with a new entry -Run the exportfs -p command
Full path to the director that is exported: /vol/vol0/pubs/ -rw=host1:host2, root=host1 /vol/vol1 -rw=host2 /vol/vol0/home
SNAPSHOTS
Set snapshot reserve to 100% in case entire volume changes (e.g. db re-index), otherwise lun/volume will go offline
Quota
You can do a quota on a group by applying a user quota to the SID of the AD group
Adding disks to a storage system You add disks to a storage system to increase the number of hot spares, to add space to an aggregate, or to replace disks. Before you begin Before adding new disks to the storage system, confirm that the storage system supports the type of disk you want to add. For the latest information on supported disk drives, see the System Configuration Guide on the NOW site (now.netapp.com). Steps 1. Install one or more disks according to the hardware guide for your disk shelf or the hardware and service guide for your storage system. For storage systems using software-based disk ownership, the new disks are not recognized until they are assigned to a system and pool. You can assign the new disks manually, or you can wait for Data ONTAP to automatically assign the new disks if your system follows the rules for disk autoassignment. For storage systems using hardware-based disk ownership, Data ONTAP displays a message confirming that one or more disks were added and then recognizes the disks as hot spare disks. 2. After the new disks have all been recognized, verify their addition, and (if your system is using software-based disk ownership) their ownership information, by entering the following command: disk show -v Managing disks | 65 You should see the new disks, owned by the correct system and in the correct pool, listed as hot spare disks. 3. (Optional) You can zero the newly added disks now, if needed, by entering the following command: disk zero spares Note: Disks that have been used previously in a Data ONTAP aggregate must be zeroed before they can be added to another aggregate. Zeroing the disks now can prevent delays in case you need to quickly increase the size of an aggregate. The disk zero command runs in the background and can take hours to complete, depending on the size of the unzeroed disks in the system. Result The new disks are ready to be added to an aggregate, replace an existing disk, or remain available as hot spares. Related concepts Guidelines for assigning ownership for disks on page 55 How ownership autoassignment works for disks on page 50 How Data ONTAP 7-Mode works with disks on page 29 Replacing disks that are currently being used in an aggregate You can use the disk replace command to replace disks that are part of an aggregate without disrupting data service. You do this to swap out mismatched disks from a RAID group. Keeping your RAID groups homogenous helps optimize storage system performance. Before you begin You should already have an appropriate hot spare disk of the correct type, size, speed and checksum type installed in your storage system. This spare must be assigned to the same system and pool as the disk it will replace. About this task If you need to replace a disk—for example a mismatched data disk in a RAID group—you use the disk replace command. This command uses Rapid RAID Recovery to copy data from the specified old disk in a RAID group to the specified spare disk in the storage system. At the end of the process, the spare disk replaces the old disk as the new data disk, and the old disk becomes a spare disk in the storage system. 66 | Data ONTAP 8.0 7-Mode Storage Management Guide Note: If you replace a smaller disk with a larger disk, the capacity of the larger disk is downsized to match that of the smaller disk; the usable capacity of the aggregate is not increased. Step 1. Enter the following command: disk replace start [-m] old_disk_name new_spare_name If you need to use a disk that does not match the speed or pool of the other disks in the aggregate, you can use the -m option. If you need to stop the disk replace operation, you can use the disk replace stop command. If you halt a disk replace operation, the target spare disk needs to be zeroed before it can be used as a data disk in another aggregate. Related concepts How Data ONTAP works with hot spare disks on page 105 How Data ONTAP 7-Mode works with disks on page 29 Guidelines for assigning ownership for disks on page 55 How ownership autoassignment works for disks on page 50 Related tasks Adding disks to a storage system on page 65 Assigning ownership for disks and array LUNs on page 57 Converting a data disk to a hot spare Data disks can be converted to hot spares by destroying the aggregate that contains them. You must convert a data disk to a hot spare before moving it to another storage system. About this task Converting a data disk to a hot spare does not change the ownership information for that disk. Step 1. Destroy the aggregate the contains the disk by entering the following command: aggr destroy aggr_name All disks in use by that aggregate are converted to hot spare disks. Managing disks | 67 Removing disks from a storage system How you remove a disk from your storage system depends how the disk is being used. By using the correct procedure, you can prevent unwanted AutoSupport notifications from being generated and ensure that the disk will function correctly if it is reused in another storage system. About this task Remember that if you are removing a disk because it has failed or because it is producing excessive error messages, you should not use the disk again in this or any other storage system. If you are removing a spare disk, and you might use the disk in a storage system running an earlier version of Data ONTAP, be sure you erase the disk ownership information from the disk before removing it from the storage system. Note: You cannot reduce the number of disks in an aggregate by removing data disks. The only way to reduce the number of data disks in an aggregate is to copy the data and transfer it to a new aggregate that has fewer data disks. Next topics Removing a failed disk on page 68 Removing a hot spare disk on page 69 Removing a data disk on page 69 Removing a failed disk A disk that has already failed is no longer counted by Data ONTAP as a usable disk. You can just physically disconnect the disk from the disk shelf. Steps 1. Find the disk ID of the failed disk by entering the following command: aggr status -f The ID of the failed disk is shown next to the word failed. The location of the disk is shown to the right of the disk ID, in the columns labeled HA, SHELF, and BAY. 2. Remove the disk from the disk shelf, following the instructions in the hardware guide for your disk shelf model. 68 | Data ONTAP 8.0 7-Mode Storage Management Guide Removing a hot spare disk Removing a hot spare disk requires you to remove ownership information and notify Data ONTAP that you are removing the disk to avoid unwanted AutoSupport messages. Steps 1. Find the disk name of the hot spare disk you want to remove by entering the following command: aggr status -s The names of the hot spare disks appear next to the word spare. The locations of the disks are shown to the right of the disk name. 2. Remove the software ownership information from the disk by entering the following commands in the specified order: priv set advanced disk remove_ownership disk_name priv set 3. Enter the following command to spin down the disk: disk remove disk_name 4. Wait for the disk to stop spinning. See the hardware guide for your disk shelf model to learn about how to tell when a disk stops spinning. 5. Remove the disk from the disk shelf, following the instructions in the hardware guide for your disk shelf model. Removing a data disk The only time you should remove a data disk from a storage system is if the disk is not functioning correctly. If you want to remove a data disk so that it can be used in another system, you must convert it to a hot spare disk first. Steps 1. Determine the name of the disk you want to remove. If the disk is reporting errors, you can find the disk name in the log messages that report disk errors. The name is prepended with the word "Disk". 2. Determine the location of the disk you want to remove by entering the following command: aggr status -r The location of the disk appears to the right of its name, in the columns HA, SHELF, and BAY. Managing disks | 69 3. If you do not need to remove the disk immediately, enter the following command to pre-fail the disk: disk fail -f disk_name Attention: You must wait for the disk copy to complete before physically removing the disk. Data ONTAP pre-fails the specified disk and attempts to create a replacement disk by copying the contents of the pre-failed disk to a spare disk. Note: This copy might take several hours, depending on the size of the disk and the load on the storage system. If the copy operation is successful, then Data ONTAP fails the disk and the new replacement disk takes its place. If the copy operation fails, the pre-failed disk fails and the storage system operates in degraded mode until the RAID system reconstructs a replacement disk. 4. If you need to remove the disk immediately, enter the following command: disk fail -i -f disk_name -i fails the disk immediately. Attention: Do not immediately fail a disk unless it is causing immediate performance or availability issues for your storage system. Depending on your storage system configuration, additional disk failures could result in data loss. The disk fails and the storage system operates in degraded mode until the RAID system reconstructs a replacement disk. 5. Remove the failed disk from
http://netapp-blog.blogspot.com/2010/01/there-were-numerous-time-when-i-wanted.htmlDefragement in NetApp Usually we face this problem with our PC and then we defrag our volumes clear temp files and what not; most of the times that solves the problem, though not fully but yes it gets better. In NetApp though we don’t have to deal with fragmented registry or temp files but due to nature of WAFL file system it gets fragmented very soon, soon after you start overwriting or start deleting and adding the data to volume. So what do you do then? Well the answer is very simple use ‘reallocate’ command. Yes, this is the defrag tool of NetApp built right in the Ontap OS. First you have to turn on the reallocation on system with ‘reallocate on’ command. This command turns on the reallocation on system and same way turns off with off switch. This can be used not only on volumes, infact you can run this on a file, lun or aggregate itself. However I should warn you that optimization of lun may not give you any performance benefit or may get worse, as Ontap doesn’t have any clue what’s in the lun and it’s file system layout. If you want to run the reallocation only one time you should use -f or -o switch however if you want Ontap to keep a track of your FS and optimize the data when if feels necessary you should control it with –i switch or schedule it with ‘reallocate schedule’ command. To check current optimization level of volume, you can use ‘reallocate measure -o ’ or if you want to feel adventurous use ‘wafl scan measure_layout ’ through advanced mode, though I don’t suggest using wafl set of commands in general use but yes sometime you want to do something different. This command is pretty straightforward and no harm (except extra load on CPU and disk) so you can play with this but you should always consider using -p switch for volumes having snapshot and/or snapmirror on to keep the snapshot size small.
Brocade Commands
debugshow - view logs chassisshow - shows serial number tsclockserver - set NTP source tstimezone --interactive - set timezone ipaddrshow - show ip address ipaddrset - set IP address switchname - set name fastboot - fast reboot reboot - reboot cfgclear - clears all aliases, zones, and zone configs cfgsave - saves curent zone config cfgenable - enable current zone config
switchshow - shows status of all ports
To Change Domain ID switchdisable configure y enter new domain id switch enable
to configure an ISL port: portcfgislmode <port #> <.1 to enable or 0 to disable>
Export a list of IPs allowed to relay through your Exchange 2003 SMTP server: Note that you have to use the DNS name of your AD server and the Exchange EVS name or server name, IPs will not work. http://support.microsoft.com/kb/935635 C:\ExIpSecurity>cscript ipsec.vbs -s exchangeservernameorEVSname -o e -r relay -d domaincontrollername
Export a list of email addresses in your AD domain using ldifde: Ldifde -d "DC=domain,DC=co,DC=uk" -r "(&(mailnickname=*))" -l proxyAddresses -f emailaddresses.txt
To test for open UDP and TCP ports you can use portqry.exe, even better is the GUI user interface which also comes with pre-defined profiles that automatically check the ports needed for AD authentication among many other options. Download it here
- Occasionally you may come across an app or batch file that requires a user to remain logged in order for it to run. You can use the resource kit utilities instsrv.exe and srvany.exe to run the app or batch file as a service. http://support.microsoft.com/kb/137890
If you have problems getting the batch file to execute from the application directory add the "appdirectory" path as seen below.
- When serving files in a download link from an IIS website that have no associated application on the web server, e.g. .oft files (outlook email template), you have to edit the server mime type so that the web server can tell the client browser which application to use to open the file. Through your server's
properties page in IIS, click on the MIME Types button. In the
MIME Types property page, if .oft is not there, click on the New...
button, enter .oft in Extension field and in the MIME Type field,
specify the MIME Type, i.e., application/octet-stream.
- If you ever have problems getting a device to install on a Windows machine or you would like to cleanup your device manager from the un-used drivers of every single hardware device you've ever connected to your computer, follow these instructions. The 'show hidden devices' option does not reveal the 'ghosted' devices by default. DEVMGR_SHOW_NONPRESENT_DEVICES
For automating tasks the Windows FOR command is a powerful tool that I've used many times:
In this first example, FOR is used with xcopy to copy a file from one computer to all computers listed in the file "list.txt". If running this from the command prompt remove the two %% signs assigned in the variable and make it just one % sign. Otherwise run this command from a batch file.
For /F %%w IN (list.txt) Do XCOPY /F /E /V /I /H /R /Y /Z d:\mike\setup.iss "\\%%w\c$\Program Files\Avaya\Contact Center Express\Desktop\Contact Center Express Desktop\"Desktop\"
Next I used it for creating 800 AD user accounts with an assigned password.
for /F "delims=! tokens=1,2,3" %h IN (c:\dsadd\file.txt) Do dsadd user "CN=%h,OU=Store Accounts,DC=MVgroup,DC=CA" -samid %i -pwd %j
File.txt was created by using excel and the =concatenate() function to merge cells. The end result file content looked like this (note the ! for separating values):
mvstore2010!mvstore2010!password600
Finally, I also used FOR to ping a list of computer names and output the results to a log file. This is an easy way of checking which computers are on or offline (assuming ICMP is not blocked by any firewalls)
for /f "tokens=1" %a in (names.txt) DO @ping -n 1 %a ^| ping.log