An EXTREMELY important part of working in IT is DOCUMENTATION. Yes, you need to know what you’re doing. But, I can honestly tell you, there are ALWAYS things that are going to be specific to YOUR installation. Yes, I know how to clone an E-Business Suite database from PROD –> DEV. But I would still have questions if I were asked to do it in your environment for the first time.
Different Types of Documentation
I would typically want to see a few different “types” or “categories” of documentation when being introduced to a new site.
Standard Documentation (should be reviewed/updated periodically)
- Environment Documentation – Typically a network diagram (Visio) or similar sketch that would show me which servers are involved in each system. This would include servers and storage, of course, but also anything “special” that we have to consider when configuring things (load balancers? special firewall rules?)
- Process Documentation – How do you perform backups? What is your cloning procedure? If you’re in a standby configuration, what is your failover procedure? How are code migrations handled?
Project Documentation
- Documentation for Specific Efforts – Unless an effort is underway, you may not have this one. Here, I’d be looking for any documentation for a particular upgrade, platform migration, etc. It makes sense that this would be more of a “living” document. We did this in PASS0, here’s what we’re going to do in PASS1 and then again in PASS2, with revisions along the way…
My Tools
There are a few tools that I tend to use when producing my documentation.
SnagIT – Screen capture/notation
Microsoft Word- Primary Documentation Format
Microsoft Excel – Used mostly to compute timing and patch lists
Microsoft Project – Used to help sequence steps in a large effort (upgrade/migration)
Microsoft Visio – System/Network Diagrams, Process Flow Diagrams
There are, of course, alternatives. You might not want to purchase SnagIT, for example. If you’re looking for an open-source alternative, I’d suggest looking at GreenShot, which has most of the same functionality.
My Approach
First, you’ll notice that I do NOT create my documentation in Microsoft Excel. I know that it is pretty common for many DBAs, but I really despise this approach. I find the documentation very difficult to decipher and very difficult to follow.
Instead, I tend to produce a Word document. This approach provides a linear path through an effort. “Begin at the beginning,’ the King said gravely, ‘and go on till you come to the end: then stop.” (the King speaking to the White Rabbit in Alice in Wonderland).
Your documentation should NOT require in-depth knowledge in order to be followed. My intent, is that any reasonably-skilled Oracle EBS DBA should be able to follow any documentation I produce. This means that I try not to make very many assumptions. I will generally assume that they know how to run a script using the sqlplus command line, know some basic Unix/Linux commands, and have a basic understanding of some of the EBS tools. But that’s pretty much it.
Getting Started
The PRIMARY thing you’ll want is a good Microsoft Word Template. A Word Template is basically just another document. Only real difference is the way that you save it (and where you save it). Start out by creating a new document using the normal “Blank Document” template. Create your own styles and layout. But I find it useful to use the built-in “Heading 1” “Heading 2”, etc. styles for the most part. (You can modify them). The nice thing about the built-in heading styles is that they can be used to create a table of contents at the front of your document.
Key Styles in the Document Template
Headings – These would be the default “Heading 1”, “Heading 2” styles as mentioned above. I do modify them, however. I like to have each level numbered, this makes it easier to reference things in other documents or when building your project plan. It also makes collaboration easier (“Please do the steps in Section 4”, for example.).
Code – This style creates a light blue shaded box with bold text and uses a fixed-pitch font (Courier or Lucidia Console, for example). This makes it easier to copy/paste (and understand) various operating system level commands in the process.
Note – This style creates a pink box with BOLD RED text. Here we will point out important things and/or warnings about items that might have “bitten” us in previous iterations.
Editor – This style creates a green box with BOLD text. I use this for various “notes to myself” about things to adjust going into subsequent iterations.
Key Sections of the Document Template
Cover/Title Page – Simple top sheet with basic information about the author, client, consulting firm, purpose of the document.
Table of Contents – Simply insert a Table of Contents using the References ribbon in Word. It can be generated automatically based on the headings in the document.
Document Revision History – Simple MS Word Table to track changes
Body of the Document – Where we record the various steps/commands/timings/comments about the process.
Reference Documents – This is a simple table with only two columns. DocID and Title. I use this to keep track of the various MOS notes that were used when researching the effort. By putting it into a table and dividing things into these two columns, it’s simple enough to sort and remove any duplicate entries.
TURN OFF AUTOCORRECT/AUTOFORMAT!!!
One of my BIGGEST complaints about Microsoft products is that they try to do too many things FOR me. Because of this, they frequently do them wrong. One of those things is “Autocorrect/Autoformat as you type”. This REALLY gets in the way of my documentation.
So, we’ll want to turn many of these items OFF.
In Word, go to File –> Options –> Proofing –> AutoCorrect Options…
There are TWO tabs you’ll want to update. “AutoFormat As You Type” and “AutoFormat”. In particular, you’ll want to turn OFF “smart quotes” and replacing hyphens with dash. Those two items will interfere with the various commands you’re trying to document and won’t work if someone uses copy/paste to execute them.
Making it a Template
Once you’ve created your basic document layout, save it as an actual template using File –> Save As and use the “Word Template (*.dotx) format. You’ll want to save it into your Templates directory so that it is easy to use for new documents.
Actually Writing the Documentation
Start with a fresh document using the template you created above.
Edit the cover page and revision history.
I also like to include an “Overview” section at the top of the document where I describe the basic concepts of what we’re trying to accomplish. Sometimes that may include various diagrams, etc. to get the point across.
Include references in the section headers that indicate which MOS document this step comes from and where (which server/environment) it should be performed on. For example, “MOS 12345.1 [dbTier]” or “MOS 12345.1 [appsTier RUNFS]”.
Use the Code style for the actual commands that need to be typed. Pay attention to line breaks (use the Unix “\” line continuation character followed by an “enter” if necessary).
NEVER include actual passwords in the document unless they are generic. Instead, define certain environment variables in a section near the top of the document and reference those instead.
export APPSPW=*****
export SYSTEMPW=*****
export WLADMIN=*****Then, when you need to use those variables later, they should be available.
. ${HOME}/apps.env run
cd ${ADMIN_SCRIPTS_HOME}
{ echo "APPS"; echo ${APPSPW}; echo ${WLADMIN}; }|./adstpall.sh -nopromptmsgMake use of standard variables wherever possible. This would include things like ${ORACLE_HOME} and ${ORACLE_SID} on the dbTier, but also ${APPL_TOP}, ${RUN_BASE}, ${ADMIN_SCRIPTS_HOME}, etc. on the appsTier.
Try to use commands that will do things in a robust manner. It is entirely possible that someone might try to do a step twice. Take that into consideration. Instead of outright deleting something, rename it. (You can go back and delete it later). For example, if you’re doing an upgrade from 12c to 19c, if you rename the 12c ORACLE_HOME to “12.1.0.2.DEL”, any references to it would be sufficiently “broken” but the files are still there for you to reference later on.
Make backups before editing critical files. I tend to use a date/timestamp to create the backup file. There are several advantages to this approach. The most obvious being, you now have a backup. But you also have a “before” file that can be used to troubleshoot things (a simple “diff” can tell you a lot). An example of a backup would be:
cp ${CONTEXT_FILE} ${CONTEXT_FILE}.`date +%Y%m%d-%H%M`Keep track of timing. There will be a lot of steps that only take a few seconds or a handful of minutes to execute. But, for anything that takes more than 5-10 minutes, I try to record a “start/stop” time and calculate an “elapsed” time at the end. Not only does this give the reader an “idea” about how long a task will take, it can help you to plan that production outage later on.
Text (ASCII) Things. You may want to include the output of a particular command in the document. ETCC, for example, produces some out put that is particularly useful in documentation. For this, I tend to copy/paste the text into the document and use the code style. You might also want to highlight certain sections that are important. If you’re including the output of something interactive (adcfgclone.pl, for example), it would make sense to identify any “responses” you provide highlight (in yellow) and use bold on those portions. In most cases, I do NOT recommend taking a screenshot of a terminal window. (Graphical images aren’t searchable and can make a document large and unwieldy, I try to avoid those wherever possible).
Graphical Things. As I mentioned above, I typically try to avoid including graphical images in the document. But, there are some situations where they make sense. For those, be sure to paste in any screenshots for steps that need to be performed using a GUI tool (RapidWiz, WebLogic Console, dbca, dbua, etc.). Use SnagIt, Greenshot, or a similar tool to capture only the necessary windows. With those tools, use arrows, highlighting, and other notations to indicate anything that needs to be done on that particular screen. Sometimes, it might make sense to number the various fields and include a numbered list explaining things below the image when you paste it into the Word document.
Other Things…
I mentioned Microsoft Excel earlier on. Personally, it’s a horrible tool for documenting things. If you really need something in a large table format (one that doesn’t fit well into a table within a document), then go to Excel. It works well for me when I’m doing patch dependency research, for example. But please, don’t use it for documentation.
The other thing that I mentioned was Microsoft Project. For me, it’s useful to produce a GANTT chart of the DBA portion of certain efforts. Upgrades, platform migrations, large patching efforts, etc. can all benefit from this. The main advantages to me are that it allows me to map out certain dependencies between tasks and also allows me to plan for certain items that might be long-running. For those, I can map out when it CAN start and when it MUST finish. This knowledge can be important in working to make the overall effort more efficient. The Word document is about the steps. The GANTT chart is about minimizing downtime.