R12 – A Detailed Transition Plan

So your UAT is heading for completion, you’ve got as many bugs out as you can and your ready for go-live.

Problem is…..this is the most dangerous part of the entire project. All the hard work and success could go be wrecked over a poorly planned weekend (or long weekend if your upgrade is really tricky or big)

Planning is the key.

It’s recommended that everything is documented very, very heavily. Start off with a PM.030 and get into your head exactly what your going to do.  And a PMP.030 Transition Plan isn’t just what you will do over the cutover weekend; its before, during and after – how are you going to support the R12 when live? Are your teams ready for that?

Get all your Functional Setup into a Spreadsheet, BR.100 per module or other installation documents. Give assignments to each person. Make it clear who does what, when they do it, order, dependencies, etc. On a cutover weekend there is ZERO time for mistakes.

Have a spreadsheet of all tasks to countdown, dates, people, etc. Mark them off one by one as you go.

Have another spreadsheet for all cutover tasks. I suggest some sort of matrix where setup is assigned and you can see each persons workload. Then try to balance that workload. If one person is too slow or too busy on the cutover, you’ll hit major delays on cutover weekend.

Your source code should be in a Source Control System (we used Subversion – it’s FREE – it’s GREAT). You should repeatedly clone Production and Test the deployment until very close to Production. Also apply Functional Setup and retest.

Are you confident that what was passed in UAT is what you’ll put in Production?

Keep doing the clone, setup and test until your happy that the answer is 99.9% yes.

Also during final approach to go-live, do a simulated financial close on the Clone. Trust me this will save you a lot of heartache. Wouldn’t you rather find disastrous stuff you need to do before you hit it in the crazy, high pressured time after go-live. Pay careful attention to AP closing. It’s tough. Run the accounting health checks.

You must have an automated mechanism for code application and your DBA’s should be familiar with this. And by automated it can be as simple as a shell script to move stuff to the right place and a script to cut and paste and compile objects. Get this 100% before you do it.

One tactic that worked well for us was to do setup on 11.5.10. We’re not saying everything, but some menus, some value sets, messages…..every little helps. Of course you can also do these on the day using FNDLOAD, but the point is the more you can do ahead, the less you have to do in a high pressured weekend.

Get the meetings going with Business. Make sure everyone is aware of how and what is getting done. We closed on R11.5.10 for all subledgers prior to R12 apart from GL. This really helped reduce overall errors and risk, as all that had to be done in week one of R12 was a GL close. It went very smoothly. And then you have 4 weeks until your first real R12 close – gives you time to simulate another close 3 weeks into your financial period. Now some may disagree with this approach and prefer to go-live after month end closing. I understand this is another perfectly valid option. In our case we made the decision to jump immediately after subledger closing as we also run two large payrolls twice a month…….so our cutover windows for making the jumps were seriously limited. Also we had done this procedure many, many times (including on very recent clones), so had a level of confidence that this was safe to do. If your cutover testing isn’t giving you that confidence, suggest you close in R11 fully then jump to R12.

Make sure you have plenty of meetings with your DBA Team, Unix Team and Management. I prefer to plan for the worst; set the expectations low and then everyone will be prepared mentally for a tough time. If it goes smooth then everyone is happy. Doing an R12 is the toughest upgrade anyone will have ever done, don’t underestimate the issues you will have. Many, many companies have had horror stories during go-live.

So your plans are done, teams are briefed and your now ready for launch. Did you actually remember in advance to tell all users (and post notices on your website internal and external) that your system is unavailable due to upgrade? Did you remember to tell all the other systems (Legacy, data warehouse, etc) that your system is unavailable?

Oracle people (including myself think the world is Oracle). Remember there’s more to life than Oracle…..and most companies have a lot more systems than just Oracle, although Oracle is the foundation…..so if you take that away and forget to announce, you’ll have some very unhappy people……

Make sure you brief your helpdesk. Make sure they know the escalation procedures. Even if you do a great job, it’s going to be a busy time. Make sure you Oracle ERP Team is also ready to work round the clock post go-live, just in case…….

We brought our ERP down at 5:00PM on a Wednesday after closing all subledgers. Actually it was 5:52PM and the DBA’s were already complaining like crazy.

The DBA’s in our case brought the system down and started the export. They worked through the night to export from IBM and bring this into HP Linux. (Yes, we throw in not only an upgrade but also a migration for fun…..)

The Upgrade started on the Thursday and was then backed up early Friday morning.

The DBA’s automatically deployed the code (7,000+ files) onto what was effectively a live Production system. (with no one else on and of course backed up on R12 AND R11.5.10  before we started).

The Deployment Team rolled in at 10AM on Friday, sat around for a few hours and then started at 11:45AM to do setup. Of course we screamed at the DBA’s for being 1hour and 45 minutes late, but they claimed we were 52 minutes late handing the system over on Wednesday…….things are always tense during an upgrade cutover.

Functional Setup continued until 9PM on Friday night (remember all those new modules, Payments is a large setup, and there are loads of System Admin tasks  – Hint split some setup into Application Developer responsibility. Do others setup using FND_LOADER. I think your System Admin may be a bottle neck – that’s what happened with us as there were just so many changes between R11.5.10 and R12)

Around 9:30PM Friday the database was ready to be backed up and cloned. DBA’s worked to clone the database and make it available at 10:00AM Saturday.

Testing Team rolled in on Saturday and did a quick Financial Reconciliation. All well and good testing started on major key functionality. I hope in your cutover plan you detailed what would be tested, who would test, etc.  In addition each Team Lead should prepare a more detailed Test Plan for the transition and brief their respective teams. Concentrate on key functionality. If you do this, you’ll be doing well if it all works on day 1. Testing continued for about 10 hours on Saturday and started at 9:00AM Sunday.

As we caught problems, we logged them and got our System Admins/DBA to deploy missing items to Production (with heavily controlled approval, logging of bugs, etc so we tracked every single change)

Around about 4:00PM on Sunday we had a Team Lead meeting. Each Lead was asked for a go/no go signal. With the decision to go made, we went to Management. The current status was reported and management after discussion gave the green light.

All workflow services were turned on (with emails being caught in our email system so we could monitor and then release – nice little trick). Once we were happy emails/workflow was OK, we released one (a Leave from one of my staff) and tried the complete cycle. Once we were happy with this, we released all workflow, all concurrent jobs (we had simulated release of all concurrent jobs on Saturday to avoid surprises) and there was no going back.

We went live on R12.0.4 (RUP5) at 5:00PM on 2nd August 2009, some 96 hours after bringing down the R11.5.10. (Although we are on RUP5 we’re really heavily patched on Financials so RUP5 is a bit misleading)

Teams had worked round the clock, across all areas to make the cutover a huge success – we finished 30 minutes ahead of planned time. Now that’s pretty good planning. Have you planned to that level? If not you should be.

On 3rd August, our users logged on to R12 and we were fully operational.

Advertisements

Tags: , , , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: