Oracle SOA Suite 12c Sandbox
Everyone who survived the complete renovation at home would agree that the second main success factor (after accepting everything what wife suggests) is the right set of tools. Moreover, tools shall be assembled into right bindle, properly adjusted, fairly reachable, and easily transportable.
Traditionally, our tools are WLS, SOA Suite and DBs - all what we use for establishing SOA technical infrastructure (TI) and bundling is done by VirtualBox - free Oracle virtual machine.
Oracle Technology Network can offer lots of pre-built VM with different tools for developers and we have a dozens of good publications (Evaluating Oracle Linux From Inside Oracle VM VirtualBox, Using the Oracle Developer Days VirtualBox Image just couple of those), dedicated to packaging and bundling SOA toolsets on VM using Docker, Packer and Vagrant (which covers the first two). Usually our clients have their own understanding of developer VMs and SOA TI and quite eager to provide their own realisations. So, naturally, the question is - why do we need yet another post on the similar topic?
There are several equally important reasons:
1. Vagrant (and subsequent tools) is amazing, but before we start isolating and packing our dev environment we have to make sure that this is exactly what we need.
2. Even if we got something suitable for development (not only for learning) from OTN, there is no guarantee that we will manage to install it using standard File -> Import Appliance. The notorious error, Result Code: VBOX_E_FILE_ERROR (0x80BB0004). There are still some workarounds for that:
- Download .ova VB image
- Create new VM of the same type and HD size (most commonly Oracle Linux)
- Set optimal memory size
- Choose "Use an existing virtual hard drive file" (ova is an archive technically and all drives and metadata - .ovf, .vmdk and .mf can be extracted using 7Zip utility)
- Complete creation process and boot.
That will work just fine, although there are so many reasons why we should avoid that.
3. VM with dev TI from our clients would not exactly represent our understanding of proper SOA TI, with many elements which we cannot control, even connection to our dedicated machine: "Connection was denied because the user account is not authorised for remote login". Some practical joke from Citrix administrators. Although is very simple, this error will take some of your precious time. Besides, we need our SOA toolkit for establishing SOA Frameworks according to generic SOA Principles, which are vendor-neutral (i.e. not bound to Oracle or IBM and so on realisations). In fact, this criterion is the main driver. We also want to separate developer tools and middleware server parts, which we quite often get in one bundle via remote desktop connection. Very slow and inconvenient combination.
4. The curiosity is equally important driver. We always can find some new combinations of newly available SOA tools in order to test the effectiveness of SOA frameworks.
5. The last and most important reason is already mentioned in pos. 3. We will build and evaluate different SOA frameworks and we need a reliable, unified and most importantly - transportable testbed.
Thus, this post is just a working log of VM creation which will be used by all teammates for establishing reusable SOA frameworks. This VM should be separated from development tools, which developer should be able to choose and configure at their convenience.
Frankly speaking most of the situations, described below can be easily avoided if you read first the publications, mentioned at the beginning of this post, or just check the documentation for the detailed installation instructions for every selected MW component on VirtualBox VM.
VM should provide all necessary server recourses to SOA and DB developers, including latest 12c DB and latest SOA Suite 12c. All resources must be accessible from outside, host OS. Optimal VM layout presented on figure below:
From this figure we can see that establishing of this VM can be divided into three distinctive parts:
- Create reliable VM Linux container with full network interoperability
- Install Oracle DB 12c with separate VM guest user and connect it to host dev tools
- Install complete SOA Suite 12c and connect it to host dev tools
And here we will start from Linux container.
1. Installation of Linux VM
Our host OS is Windows 8.1 and our plan to install as a guest the latest (at the moment of writing) Oracle Linux server 7.1. Irrelevantly to the host/gest versions and bits the first problem you might hit right after VB installations is that you cannot create/boot new VM. The error you get could be quite deceptive at first.
You 64-bit guest will fail to detect a 64-bit CPU and will not be able to boot
Reading further we will find that this error is not related directly to the 64bit processor options of host/gest (although it must be supported by guest and host). No matter how powerful your physical machine is, the processor virtualisation technologies VT-x or AMD-x could be disabled by vendor by default. Thus, before start any exercises it would be prudent to check our BIOS settings. Run msiinfo32 Windows utility and search for “virtualization” parameter, please see the screenshot below:
If you have CCleaner from installed then you might find useful another utility from piriform – Speccy which can provide much broader information than msiinfo32, including processor core temperature. One way or another, most probably you will find that the virtualization is disabled in firmware and the only way to change it is to go to the BIOS settings during booting process (press and hold F10 or DEL keys, depends on hardware). Location of this parameter could be rather unpredictable, not always under “Processor” menu section. Usually it can be found under Security option. After fixing this parameter the installation is straight forward for Linux. After completing installation do not forget to install guest additions from mounted CD.
Before address tasks 2 and 3 we must ensure that VM internetworking is capable support:
- Access from host to guest over SSH (putty and sqlplus traditionally)
- Access from host to guest from our preferred dev tools (JDev, SQLDev)
- Access from guest hosts folders (install folder for .tars, .gzip and rpm packages and connection to local subversion)
- Access internet from guest OS for critical Linux updates and general yum command execution
- It might happen that one VM would not be enough for us, so we will need to establish a bridge between different virtual boxes.
For these internetworking and bridging purposes we will need to configure two adapters – one NAT and one Host-Only Ethernet, please see figure below:
At this moment we use IP4 settings by default, which is not good for the last requirement – VM bridging, but fixed addresses can be assigned according your network preferences. Actual IPs will be presented in all further installation and configuration DB and OFM steps, but for now we would like to focus on port forwarding rules for DB and SSH ports, see screenshot above.
After installation of guest additions you will get not only bidirectional drag-n-drop, shared clipboard and seamless mouse integration, but also ability to mount your host folders as a shared resources.
It’s quite handy for further installations previously downloaded modules and applications. Declare your shared resource in VM settings as presented below and then mount this folder for current Linux user as
sudo mount -t vboxsf orasetup /home/oradev/Utils/shsetup
If you select Auto-mount option for your shared folder, please bear in mind that by default host shared folder will be mounted to /media/sf_name-of-vbox-host-shared-folder, as you can see on figure below for shared setup folder.
Thus, please do not try to add mount –t command to the /etc/profile or /etc/rc.local or your user .bashrc scripts or add vboxsf to Linux loadable modules. At best it will have no effect, worst – you will have to manually trace and fix your booting sequence. All what you might need to do is to create symbolic link to media resource and give a proper link name.
As described on our first figure, we must create two users, oradev and oracle as a prerequisite for further installations and add them to the particular groups: oinstall, oper, oracle, dba, SYSDBA, OSDBA, OSOPER. Linux command useradd and groupadd will do the trick or you can use user and groups manager utility if you have this option installed, please see figure below:
There is yet another mandatory prerequisite for all users and all further installations, requiring the presence of proper java platform and java development kit (i.e. JDK, that’s important, just JRE, runtime environment is not enough). Naturally, all further installations will be as java –jar <package_name.jar> With Oracle Linux 7 you will get OpenJDK JVM, which is perfect JDK for generic Java development, if you try to install any oracle package using it you will get something like
OpenJDK 64-Bit Server VM is an unsupported JVM
That means that we need to install vendor-specific JVM/JDK, and this JDK is not bundled neither with WLS, DB or OFM, so must be installed separately. For better compatibility we will use version 7, (8 is available, but compatibility should be put before our curiosity).
Please use whereis, whatis and java – version Linux commands to identify the current JDK and java versions and set with grep <keyword> for validating current JAVA_HOME settings. Download correct JDK and install it, it’s rather straightforward. Make sure that JAVA_HOME is correctly declared for all users.
As the result of the first step we should have healthy Oracle Linux 7.1 server with two users – oracle and oradev, each belongs to the required groups, visible in terminal window on preceding screenshot.
Please make sure that you dedicated enough memory, kernel and disk resources for further installations. All limits are clearly described in DB 12c Installation Guide, which you will need for the next step.
2. 12c DB Installation, oracle user
With proper JDK on place and all hardware and software requirements fulfilled, installation is just moving from one screen to another in graphical mode of Oracle Universal Installer (OUI is recommended): ./runInstaller. For our purposes we will select simplest configuration options, no ASM or Grid, but if you want to play with them set your preferences. The common requirement for all type installations is to follow OFA directory structure specification. Set ownership for the directory in which Oracle DB installed to oinstall using chown command and set the right access chmod –R 775 for all group members. From preceding screenshot you can see that 12c is installed in oracle root, but we have a reason for it. Jumping ahead we can mention that 12c CBD is not suitable for RCU installation (demonstrated in the step 3), thus, /u01 folder we will dedicate for XE installation, used for RCU. You can keep them both in product subfolder anyway.
After completing installation set the right parameters into your user .bash_profile or .bashrc for ORACLE_BASE, ORACLE_HOME (primary – 12c), ORACLE_SID. Time to check access to DB from host OS. As we are planning for two versions DB and two HOMES, we configure two ports, 1522 for ORCL and SSH port mapping to 2222, please see screenshot below.
Please make sure that these ports are opened in firewalls on host and guest. After all these preparation with some luck we could get an error, presented below –“got minus one from a read call”. It seems that DB Server do not want to accept connections from our IP. Connection configurations can be changed in SQLNET.ora . Just turn off TCP validation (set no) as demonstrated on figure below.
After that we should be able to connect to ORCL instance using our local SQLDeveloper and PUTTY/sqlplus. We are not over with databases yet, but we will continue with OFM installation as oradev user.
3. 12c SOA Suite installation, oradev user
For accessing ORCL DB from oradev we would like to install DB Client with sqlplus.
To be able to run sqlplus we have to set and export LD_LIBRARY_PATH in .bashrc as its presented in the preceding figure.
Oracle SOA Technical infrastructure includes Servers (WLS) running various service engines (BPEL, Service Bus as Unified Runtime environment) which in their turn depend on configuration parameters and functionality, provided by repository, maintained in database. Thus, any OFM installation should start with running RCU. Here what will happen if you will try to install RCU on 12C directly on 12c, as we mentioned earlier.
Not good at all, we (or I) should read the documentation more intently. The second issue, indicated on preceding - screenshot encoding itself is not a problem as NLS_LANG we can fixed in Lunix using setenv American_America.UTF8 command and bounce DB.
So, in anyway we have to proceed with Oracle XE DB installation, as we mentioned above - it will give us a fair level of isolation between main 12C Db(transparent data encryption is particularly interesting), dedicated for experimenting with new Oracle multitenant DB and OFM repository, holding key elements of SOA Infrastructure. These schemas, created in RCU we also prefer to have in isolated DB as we will investigate different ways and methods of BPEL DB cleanup and maintenance (BPEL dehydration store purge scripts).
Since OFM 11g we know that XE DB is not ready for RCU installation by default. We have to adjust number of parallel processes and opened cursor (process, open_cursor, 600 is good enough for both), using alter system set <parameter>=<value> scope=both. After that we can safely procced with RCU installation.
Preceding figure demonstrate list of components, we chose for installation with DEV prefix (on the right) and SQLDeveloper window with connection to the newly established schemas (on the left), established from the host OS. This connection was ensured by properly configured and running DB LISTENER (terminal window in the middle).
Speaking of which, LISTENER and TNS configuration must be carefully assessed as we would like to have two database accessible not in guest, but from the host OS as well, please see the lsnrctl status on figure below.
For better understanding, let’s return to the first figure of this post – we have two database servers so we have two DB_HOMES, we have several databases. Configuration, presented on preceding picture insure required connectivity. If you prefer graphical configuration utility, please use netca.
Now we are ready for the OFM installation.
The first step is to prepare SOA infrastructure.
As a result we will get WebLogic core AppServer, Coherence cache, and all essential services including TopLink, Maven, Jersey JAX-RS, JDBC drivers and management tools
After that we are ready to complete the OFM installation and create our development domain, as demonstrated on the first figure. SOA Suite and BPM management software can be installed by executing java -jar fmw_126.96.36.199.0_soa.jar. Installation steps are clearly explained in every installation screen. After completing this step we will have yet another ORACLE_HOME, now congaing all our middleware: /home/oradev/Oracle/Middleware/Oracle_Home. Inventory of our new middleware HOME we can check using viewInventory.sh from /oui/bin subfolder, see screenshot below:
Please bear in mind that depending on selected components during installation list can be quite long. Using –output_format and output_file utility option we can dump this information in a format, convenient for inspection. This is also useful when you asked to provide detailed OFM info during technical assistance process.
Domain is our workhorse and we must equip it with all harnesses, essential for delivering required result. It will be supplied with runtime environments, servers and server groups (clusters) and tools to control these groups. Just start ./config.sh from created
Middleware _Home/oracle_common/common/bin and select SOA Suite template as we need the workhorse of this type. Feel free to augment domain options in addition to mandatory, and create at list one managed server, as using admin servers for installation is not good idea.
After successful domain creation you will supplied with information about domain home and admin server console URL, please see preceding figure. At least three things we should keep under control - Node Manager, Admin Server and Managed Server. By the way, similar to configuring hardware, when you create managed server you must associate it with Machine. If you have missed this step during domain creation/configuration, you can do that from admin console and associate server and node manager to newly created machine.
Now we can start all our OFM servers, created in previous step. Sequences, Environments scrip config and WLST usage in start/stop routines explained here in great details:
All control scripts are in $DOMAIN_HOME/bin. Start Node Manager first using a)
c. $DOMAIN_HOME/bin/startManagedWeblogic.sh <server_name> <admin_server_url>
Make sure that its listening the designated port (5556 by default). After that you can start admin server b). Wait until server enter RUNNING mode and after that you can start our managed server. It can be done using WSL console as demonstrated on figure below, Control tub, or using command line c).
So, when all up and running we can proceed with final phase of VM configuration - Connect JDeveloper to App server. Start JDeveloper 12c (we assume that you have installed Oracle SOA Suite Quick start for Developers in your host OS), go to the Window menu option and select Application servers (Ctrl+Shit+G). In addition to the Integrated WLS create new App Server Connection (right mouse click), just following the screen instructions. For selected server type (Standalone WSL 12.x), you will have to provide credentials, established in previous step, host name, two ports and domain name. If everything is correct, the last configuration step will be as on screenshot below.
So we done with establishing full fledged development environment on VirtualBox, hitting almost every wall along the way. We hope that you will find this experience useful. Some scenarios were skipped, like forgotten/incorrect WLS domain admin password and related recovery steps. Do not think that this scenario is unrealistic - WLS password strength cannot be compromised, unlike DB password. Thus, if for sys DB admin you can have an "oracle" password, for weblogic admin you will have to invent something a bit more complex. Incorrectly inserted several times this password can lock admin console for certain time. Yes, radical steps like re-creation of WLS domain will solve this problem ultimately, but there are some recovery steps, which allow you to avoid this hard resolution. May be we will return to this point in some further updates of this post. The main point of this exercise was to set up some sandbox for further establishing SOA framework and evaluation generic SOA patterns on 12c platform. Thus, the next post will be dedicated to roles of Frameworks in agile SOA development.