Dear friends,
To be sure that we are discussing about the same concept and network definition you will find below some very valuable information about the Security and Network information and configuration of Exadata Cloud@Customer.
This is a very HOT topic where I would give more insight to our actuals and future customers.
Exadata Cloud@Customer Network Architecture Overview.
Network configuration view:
https://docs.oracle.com/en/engineered-systems/exadata-cloud-at-customer/eccid/exacc_overview.html
https://docs.oracle.com/en/database/oracle/oracle-database/23/aliad/iad_home.html
Your databases run on one or more virtual machine (VM) clusters that reside on Exadata infrastructure in your data center.
Exadata Cloud@Customer also includes 2 x dedicated control plane server (CPS) infrastructure that connects to an OCI region for cloud automation and administration.
OCI is Oracle Cloud Infrastructure: https://cloud.oracle.com/
- You will manage your Exadata Cloud@Customer from your OCI tenant or your secured web page.
- Your client applications and administrators connect to the VM clusters through client and backup networks that you create.
- You access your databases through standard Oracle database connection methods, such as Oracle Net.
- You access the VM clusters through standard Oracle Linux methods, such as token-based Secure Shell (SSH).
- Your Exadata and Autonomous Database administrators can use the web-based OCI Console, command-line interface (CLI), and REST APIs to connect to your OCI tenancy over an HTTPS connection.
- Your tenancy sends service management API calls to the Oracle service tenancy.
- The Oracle service tenancy connects to the Oracle-managed admin virtual cloud network (VCN).
- An outgoing, persistent, secure automation tunnel connects the CPS infrastructure in your data center to the Oracle-managed admin VCN in the OCI region for delivering cloud automation commands to the VM clusters.
For Autonomous Database, separate persistent automation and temporary operator tunnels are created, targeting separate service tenancy administrative VCNs.
If you’re using both Oracle Exadata Database Service and Oracle Autonomous Database Service, you’ll have two persistent tunnels and, as required, up to two temporary tunnels. These separate tunnels enforce a strict separation of duties for developer operations personnel, who only have authorization to take action on specific VM cluster resources, segregated by service type.
What are the Network Requirements for Oracle Exadata Database Service on Cloud@Customer
To provide secure and reliable network connectivity for different application and management functions, Exadata Database Service on Cloud@Customer uses different networks.
The following list outlines the minimum network requirements to install an Exadata Database Service on Cloud@Customer system:
Exadata Cloud@Customer Service Network:
These network will be set up to Oracle specifications and should not be modified by customer without Oracle agreement.
Control Plane Network:
Control Plane Connectivity Considerations:
In order for the control plane to function, the control plane server must be able to connect to certain Oracle Cloud Infrastructure (OCI) addresses. You must enable TCP port 443 outbound access to the endpoints in a specific OCI region as follows:
Note that the Control Plane Server must be able to establish TCP Port 443 outbound access only.
While outbound connections on Port 443 must be allowed, TCP Port 443 inbound access is not required, and it may be desirable from a security standpoint to block inbound connections.
(Functionally, bi-directional traffic is still possible over the connection once the secure outbound connection is established.)
The Control Plane Server requires customer DNS and NTP services to be functional. Minimum bandwidth requirements for the Control Plane Server internet connection to OCI are 50/10 mbs download/upload.
Some customers have security policies requiring the use of proxies for all internet connections to IT infrastructure. Customer HTTP proxy, for example, passive/corporate proxy supported for the Control Plane Server connection to OCI.
Customer
HTTPS, challenge proxy, and traffic inspection are not supported.
If you are using IP address filtering based firewall rules, due to the dynamic nature of cloud interfaces, you must allow traffic with all the relevant IP CIDR ranges associated with your OCI region as identified.
Network Interfaces Connection View
InfiniBand or RDMA Over Converged Ethernet (ROCE) Network
This network connects the database servers, Exadata Storage Servers, and control plane servers using the InfiniBand or ROCE switches on the rack. Each server contains two InfiniBand network interfaces (IB0 and IB1) or ROCE interface (re0 and re1) that are connected to separate InfiniBand or ROCE switches in the rack. Primarily, Oracle Database uses this network for Oracle RAC cluster interconnect traffic, and for accessing data on Exadata Storage Servers.
This non-routable network is fully contained within the Exadata Cloud@Customer rack, and does not connect to your corporate network. However, because the control plane servers are connected to the InfiniBand or ROCE network and to your corporate network, the IP addresses that are allocated to the InfiniBand or ROCE network must not exist elsewhere in your corporate network.
Administration Network
This network connects Exadata Database Service on Cloud@Customer servers and switches to the two control plane servers that are located in the Exadata Database Service on Cloud@Customer rack. It facilitates customer-initiated operations using the Oracle Cloud Infrastructure Console and APIs. It also facilitates monitoring and administration of the Oracle-managed infrastructure components in Exadata Database Service on Cloud@Customer.
This network is fully contained within the Exadata Database Service on Cloud@Customer rack, and does not connect to your corporate network. However, the Exadata infrastructure is indirectly connected to your corporate network through the control plane servers. This connection is required to provide Domain Name System (DNS) and Network Time Protocol (NTP) services to the Exadata infrastructure. Therefore, the IP addresses that are allocated to the administration network must not exist elsewhere in your corporate network.
Each Oracle Database server and Exadata Storage Server has two network interfaces connected to the administration network. One provides management access to the server through one of the embedded Ethernet ports (NET0). The other provides access to the Integrated Lights-Out Management (ILOM) subsystem through a dedicated ILOM Ethernet port. Exadata Database Service on Cloud@Customer is delivered with the ILOM and NET0 ports connected to the Ethernet switch in the rack. Cabling or configuration changes to these interfaces are not permitted.
Customer Network:
Customer-owned and managed networks required for the Exadata Cloud@Customer data plane to access related systems.
Client Network:
This network connects the Exadata Cloud@Customer database servers to your existing client network and is used for client access to the virtual machines. Applications access databases on Exadata Database Service on Cloud@Customer through this network using Single Client Access Name (SCAN) and Oracle Real Application Clusters (Oracle RAC) Virtual IP (VIP) interfaces.
The client access network uses a pair of network interfaces on each database server, which are connected to the customer network.
Backup Network:
This network is similar to the client access network, as it connects the Exadata Database Service on Cloud@Customer Oracle Database servers to your existing network. It can be used for access to the virtual machines for various purposes, including backups and bulk data transfers.
Like the client network, the backup network uses a pair of network interfaces on each database server, which are connected to the customer network. Physically connecting the backup network to a customer network is required.
If the customer’s on-premises storage (NFS or ZDLRA) is to be used exclusively as a backup destination for databases, then no external connectivity to OCI is required for the backup network.
Exadata Cloud@Customer also offers an Oracle-managed cloud object storage backup destination. If Oracle’s Object Storage Service is to be leveraged as a backup destination for database backups, then ensure that the backup network can reach the Object Storage Service through external connection. You must enable TCP port 443 outbound access for the backup network as follows:
Oracle Cloud Operations
Oracle Cloud Operations can use a console or REST APIs over an HTTPS or SSH connection to manage the infrastructure.
Oracle operator SSH connections travel through bastion and management servers, the admin VCN, and an outgoing, temporary, secure operator tunnel from the CPS to the admin VCN. Oracle operator HTTPS connections travel to the infrastructure through the Oracle service tenancy, admin VCN, and the outgoing, persistent, secure automation tunnel.
Oracle operator SSH connections travel through bastion and management servers; the admin VCN; and an outgoing, temporary, secure operator tunnel from the CPS to the admin VCN. You can optionally control authorization of SSH access by Oracle operators by registering infrastructure resources with a separate Operator Access Control.
Establish a Secure Connection Between Your CPS and OCI Using OCI’s FastConnect Service
Consider the solution outlined below, which leverages OCI’s FastConnect service, if you want additional isolation for the connection between your CPS and OCI in addition to the default TLS tunnel approach.
For more information, see Oracle Cloud Infrastructure FastConnect.
Oracle Exadata Database Service on Cloud@Customer service supports the public or private peering connectivity model of FastConnect.
- Fast Connect option for ExaDB-C@Customer
- Network information ExaDB-C@C
- Exadata Database Service on Cloud@Customer Administrator’s Guide
What is Oracle Operator Access Control for Exadata Cloud@Customer ?
Operator Access Control OpCtl is an Oracle Cloud Infrastructure (OCI) privileged access management (PAM) service for Exadata Cloud@Customer (ExaC@C). OpCtl provides customers with software interfaces in the customer’s tenancy to control and govern if, when, and how Oracle Cloud Operations (Cloud Ops) staff can access the Oracle managed infrastructure in the Oracle tenancy of the ExaC@C service.
These controls are a standard part of the ExaC@C service and are available at no extra cost to Oracle customers.
Operator Access Control “OpCtl” includes the following preventive security control features:
- Oracle staff access only when authorized by the customer and only for a specific Oracle work request.
- Oracle staff access is limited to explicitly approved components related to a stated and specific work request
- Oracle staff access is temporary, and is automatically revoked after the authorized task is completed.
- Customer control over when Oracle staff can access infrastructure.
- Software control over privilege escalation by Oracle staff.
OpCtl includes the following detective security control features:
- Customer notification when Oracle staff need to access infrastructure.
- Individually identifiable audit logging of every command and keystroke executed by Oracle staff.
- Customer security monitoring of all commands and keystrokes entered by Oracle staff.
- Oracle-supplied record of the Oracle staff identity to the customer when required for any command executed.
- Oracle security staff monitoring of all Oracle Cloud Ops staff activities.
- OpCtl includes the following responsive security control features:
Customer control to terminate Oracle staff access and all processes started by Oracle staff at any time. - Oracle security staff control to terminate Oracle staff access and all processes started by Oracle staff at any time.
More information on Operator Access Control for Exadata Cloud@Customer or ExaDB-C@Customer
Security staff evaluating OpCtl should also review the related documentation that describes additional controls in the
ExaC@C and ADB-D on ExaC@C services and the Oracle Cloud Infrastructure control plane, including the following:
- https://www.oracle.com/a/ocom/docs/engineered-systems/exadata/exadata-cloud-at-customer-security-controls.pdf
- https://docs.public.oneportal.content.oci.oraclecloud.com/en-us/iaas/exadata/doc/exacc-secguide.html
- https://docs.oracle.com/en-us/iaas/autonomous-database/doc/security-features-adb-d.html
- https://www.oracle.com/a/ocom/docs/oracle-cloud-infrastructure-security-architecture.pdf
- https://docs.oracle.com/en-us/iaas/Content/Security/Concepts/security_guide.htm
- https://www.oracle.com/corporate/security-practices/assurance/
- https://www.oracle.com/corporate/security-practices/corporate/security-incident-response.html
- https://www.oracle.com/a/ocom/docs/corporate/data-processing-agreement-062619.pdf
- https://www.oracle.com/corporate/contracts/cloud-services/contracts.html#online
To be continued…
