Comments

  • Jon-Eric Eliker

    Hi Arunkumar.

    Richard's suggestion is sound if you are intending to monitor custom APIs and services. However, if you are asking to monitor OCI inherent API endpoints (I am not certain based on the wording of your post)  I suspect that achieved through the OCI health dashboard here:

    https://ocistatus.oraclecloud.com

    Given the crucial role of the OCI APIs in general OCI functionality worldwide, I suspect that dashboard considers API state and many other factors when reporting "good." This may not be the level of detail you hope to see but may provide the good/bad indicator you are (potentially) hoping to see.

    I hope you find this useful.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Hi Sneharaj.

    You have encountered a commonly frustrating circumstance. That is, you must choose the "VCN and Internet" DNS option to properly provision your Database System but desire to use on-prem. DNS resolution for purposes like Active Directory. Fortunately, this has been partially addressed in a recent update to the Oracle Cloud Infrastructure service!  As of early November, you can now select to use "Custom DNS" for Subnets that contain database systems Before that, you could fool the system by having VCN and Internet only DNS during Database System provisioning then switch to Custom after. Now we can officially have this Custom setting. See here for details:

    Network Setup for DB Systems > Custom DNS Resolver

    Note there are some caveats related to this configuration mentioned in the documentation link above (for example this is only valid for non-RAC configurations).

    If you have not had opportunity to explore the two choices for DNS resolution (VCN/Internet or Custom) see here for an overview of each:

    DHCP Options > Overview

    A third option is to consider a "hybrid" approach by which you retain the automatic resolution of "oraclevcn.com" host names while introducing your own DNS servers to manage non-OCI name resolution.  It's a more complex configuration but offers great flexibility when implemented. In brief, this solution involves hosting a DNS "proxy" in OCI that forwards requests to the OCI internal DNS when appropriate and to your DNS servers (i.e. your Active Directory controller) when appropriate.  You need to take caution to avoid delays/lag in the DNS requests which is described in this blog post I shared on the subject:

    Prevent Latency in OCI-Hosted DNS Solutions (Mythics.com)

    Note there are three Oracle-hosted blogs referenced at the top of my blog post that cover the basics of configuring this Hybrid DNS model.

    I hope you find this input useful.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    I'm glad you found your solution!

    I suspect it is a policy issue as you describe especially if you are using an "Administrator" account that has full access in all compartments (including the root).  Though some of the OCI Console is good about "hiding" features you are not allowed to perform when a policy prevents such action, others will simply fail within the UI similar to what you show here. Another example of that is if you select an existing Boot Volume to use for launching a new Compute instance (i.e. perhaps you cloned an existing Boot Volume) then you select a different Availably Domain than that containing the Boot Volume you are launching with, you'll find the "launch" UI generally lets you do everything as you intend but when you launch you'll get an odd message about not finding your Boot Volume (which is true once you change Availability Domains).

    I'm sure behaviors like what you are seeing and what I describe will get worked out as the product continues to evolve.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Hi Steve. I have seen what you are seeing on occasion (or at least similar behavior). I have chalked it up to having to move in and out of multiple cloud accounts throughout the day (per the nature of my work). However, I've found that opening a private browser window (or "incognito" as Chrome calls it) has reduced flakiness like this for me close to 100%. If you haven't yet tried a private/incognito window I'd suggest that at least to help resolve your troubles.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Hi Enrique

    I am not aware of a monitoring tool or otherwise "self-service" method of tracking Service Limit changes. However, I've had success using either of these paths:

    • Reply to the "reply-to" address on the configuration email (oci-limits_us@oracle.com for me)
      This has provided rather quick response in my experiences.
    • Open an SR using the "Service Limits" category
      This is the same category that was previously required for requesting Service Limit changes. Since the option is still listed, you should expect SRs tagged with this will be routed to the right team. Again, I've used this method for checking status with success. The SR method also provides a better "paper trail" than email documenting when you requested status and "formally" requiring some type of response.

    I have been told that some of the Service Limit changes require longer because of coordination across multiple teams. For example, Reserved IP addresses, OKE, KMS, and others will likely take much longer (a week or more on occasion in my experience) than single-service needs like Compute, Block Storage, Networking, and so on (which I've seen complete within hours).

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Hi Julian.  The Bash IFS (internal field separator) variable by default will separate the input on spaces. You see that behavior here (assuming object name "a file here.txt" in Object Storage:

    for o in $(oci os object list --bucket-name frombucket --prefix 'test_image' --query 'data[*].name' | jq -r '.[]')
      do echo $o
    done
    
    a
    file
    here.txt
    

    You can overcome that by setting IFS to a new line like this:

    IFS=$'\n'
    for o in $(oci os object list --bucket-name frombucket --prefix 'test_image' --query 'data[*].name' | jq -r '.[]')
      do oci os object copy --bucket-name frombucket --region us-ashburn-1 --destination-bucket tobucket --source-object-name "$o"
    done
    unset IFS
    

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    KM81,

    Perhaps the reason is clear from the solution Kumar shared, but I'll explain (for others perhaps) that the solution was required because Security Lists govern communication among systems (IP addresses) within the Subnet as well as from outside the Subnet. I recall being surprised by this behavior since the Security List seems to exist on the border of the Subnet (thus controlling access from outside).

    As you progress through your work you may find it beneficial to establish Network Security Groups in place of the more broad Security List updates. For example, if you have three systems within a Subnet:

    • A Database System
    • A web server
    • An application server

    …and you need the application server to communicate with the Database System on TCP port 1521, you could add a rule to the Security List for the Subnet allowing traffic to 1521 for your Subnet (following the pattern Kumar shared but with TCP, destination port 1521, and the CIDR of your Subnet). Now, if you added another Database System you would indirectly have allowed the same communication from the application server to the new Database System because Security List does not constrain by destination (i.e. you allowed 1521 to any host in the Subnet).

    As an alternative, you could define these Network Security Groups within your VCN:

    • AppServer
    • AppDBSystem

    …then add your application server to the AppServer group and add your Database System to the AppDBSystem group.  Finally, create a security rule on the AppDBSystem group like this:

    • Direction: ingress
    • Source Type: Network Security Group
    • Source: AppServer
    • IP Protocol: TCP
    • Source Range: leave blank
    • Destination Range: 1521
    • Description: DB connection from App Server

    Now you will have the same result of allowing DB connections from the application server (and not allowing DB connections directly from the web server) to the Database System without adding a rule to the Security List. At this point you have achieved the exact same results. Where the NSG approach is superior is when you now add the second Database System like I shared in my example.  Assuming you do not add the second Database System also to the AppDBSystem group, you will not be able to connect to the new DB Systems from the application server.

    Where I have found the approach useful is when you have a DEVELOPMENT collection of hosts and a TEST collection of hosts within the same Subnet. You may want to ensure that connections from a DEV host to the TEST database are not intentionally made (and the same from TEST to DEV).  By having NSGs like DEVApp, TESTApp, DEVDB, TESTDB with rules in each "DB" NSG to only allow 1521 from its corresponding NSG (either TESTApp or DEVAPP) you have that assurance.

    I hope you find this explanation useful.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    I understand your frustration.  The Compartment/VCN/Subnet steps are necessary simply because you are working within the Infrastructure service which, by definition, gives you control of the core elements in the environment. I could see a place for some "quickstart" configuration that gives you a basic setup in favor of deploying and testing your code quickly.  Short of that, see the below Terraform code that should give you the basic infrastructure you need for testing Functions like I describe above.  You can use this via Resource Manager in OCI (in the main menu like your original screenshot above) by following these steps below. Note that I assume you are using an administrator account in OCI (which is the case if you are using the first/only account created for you when you started the trial):

    1. Save the Terraform script below as a since file (call it fn_demo.tf for example)
    2. Zip that file (assume you will now have fn_demo.zip)
    3. In OCI Resource Manager, create a new Stack specifying a Name (which will be suggested if you select the zip file first), Description, and attaching the zip file you created here
    4. On the next screen for the Stack, you'll see prompts for the variables I've defined in the Terraform code
      • Tenancy OCID should default to your OCI account so no change needed
      • Change the Compartment name if you'd like. Remember that a Compartment is a management security structure in OCI and doesn't affect network connectivity or interactions among infrastructure that you deploy. All that to say, the name of the Compartment isn't that exciting and relevant at this stage in the work you're doing
      • Also, the policy name is a variable for you to change. Probably not something you need to change (but I did in when testing so I made it a variable)
      • The CIDR range for the network being defined by this script may be something you want to alter. I picked a rather simple 10 address but use 200 in the second octet hopefully to make it easily co-exist with other networks you may have already defined while testing (assuming you would have selected 10.0 or something similar for any networks you already defined). Again, change this if you'd like but keep it at /16 for purposes of other assumptions in the remainder of the script
      • Finally, set the app name to something else if you wish.  For Functions, the application is a collection of functions so the name of a "real" application would probably be more descriptive that "hello-world." Here I am assuming your first app may only contain one function.
    5. Complete the Stack definition then, from the Stack details page, use "Terraform Actions" to plan your code (the default plan name suggested is fine)
    6. Once the Plan is complete, click back to the Stack (look at the breadcrumbs navigation near the top where you see "Job Details" last and click to the stack left of that) and use "apply" from the "Terraform Actions" menu. The default name and "automatically approve" are appropriate choices here as well.

    If all goes as planned, you should find a new VCN under Networking in the OCI main menu.  You will need to choose the new compartment (using the Compartment selector in the lower left on the VCNs page) you created with the script in order to see the new VCN listed.

    Hopefully this is useful should you choose to press ahead testing Oracle Functions. Best wishes!

    Jon-Eric
    ​Mythics, Inc.

    --- code below to copy to a new file ---

    variable "tenancy_ocid" {}
    variable "compartment_name" {
        default = "FnDemo"
    }
    variable "policy_name" {
        default = "FnService"
    }
    variable "vcn_cidr" {
        default = "10.200.0.0/16"
    }
    variable "fn_app_name" {
        default = "hello-world"
    }
    
    resource "oci_identity_compartment" "demo_compartment" {
        compartment_id = "${var.tenancy_ocid}"
        name = "${var.compartment_name}"
        description = "For demonstration of Oracle Functions"
    }
    
    resource "oci_identity_policy" "root_demo_policy" {
        compartment_id = "${oci_identity_compartment.demo_compartment.compartment_id}"
        description = "For Fn Service access to tenancy resources"
        name = "${var.policy_name}"
        statements = [
            "Allow service FaaS to read repos in tenancy"
        ]    
    }
    
    resource "oci_identity_policy" "compartment_demo_policy" {
        compartment_id = "${oci_identity_compartment.demo_compartment.id}"
        description = "For Fn Service access to compartment resources"
        name = "${var.policy_name}"
        statements = [
            "Allow service FaaS to use virtual-network-family in compartment ${oci_identity_compartment.demo_compartment.name}"
        ]    
    }
    
    resource "oci_core_vcn" "demo_vcn" {
        cidr_block = "${var.vcn_cidr}"
        compartment_id = "${oci_identity_compartment.demo_compartment.id}"
        display_name = "Demo VCN"
        dns_label = "demo"
    }
    
    resource "oci_core_internet_gateway" "demo_ig" {
        compartment_id = "${oci_identity_compartment.demo_compartment.id}"
        display_name = "Internet Gateway"
        vcn_id = "${oci_core_vcn.demo_vcn.id}"
    }
    
    resource "oci_core_default_route_table" "demo_route_table" {
        manage_default_resource_id = "${oci_core_vcn.demo_vcn.default_route_table_id}"
        route_rules {
            network_entity_id = "${oci_core_internet_gateway.demo_ig.id}"
            cidr_block = "0.0.0.0/0"
        }
    }
    
    resource "oci_core_subnet" "demo_subnet" {
        cidr_block = "${cidrsubnet(oci_core_vcn.demo_vcn.cidr_block, 8, 1)}"
        compartment_id = "${oci_identity_compartment.demo_compartment.id}"
        display_name = "Public Subnet"
        vcn_id = "${oci_core_vcn.demo_vcn.id}"
        prohibit_public_ip_on_vnic = "false"
    }
    
    resource "oci_functions_application" "demo_app" {
        compartment_id = "${oci_identity_compartment.demo_compartment.id}"
        display_name = "${var.fn_app_name}"
        subnet_ids = [
            "${oci_core_subnet.demo_subnet.id}"
        ]
    }
    
    output "comp_id" {
        value = "Compartment id is ${oci_identity_compartment.demo_compartment.id}"
    }
    

    -- code above to be copied to a new file --

  • Jon-Eric Eliker

    I'm seeing now that the end of my post was somehow lost.  Here is a link for a Functions tutorial that you might find interesting:

    Oracle Functions: Set up, creation, and deployment

    Jon-Eric
    Mythics, Inc.

     

  • Jon-Eric Eliker

    Hi John.  You are correct that there are still some outdated tutorials out there related to Oracle Cloud services which has changed substantially in the past few years.  You should find the information on https://cloud.oracle.com and https://docs.cloud.oracle.com/iaas/ to be up-to-date regardless what you might find through open Internet searching.

    https://cloud.oracle.com
    Here you'll find links to almost everything related to Oracle Cloud services.  You'll find the Major divisions "Applications," "Platform," "Infrastructure," and "Resources" at the top. You'll probably be most interested in what you find in Infrastructure (related to OCI) and Platform (services that leverage OCI such as the Developer Cloud Service).

    https://docs.cloud.oracle.com/iaas/
    This is the main documentation set for OCI.

    What likely matches your interests (as I read from your post) will be one of the following:

    • OCI Container Engine for Kubernetes ("OKE")
      Use this to launch and manage Docker instances controlled via Kubernetes.  OCI does much of the setup for you leaving you to create, deploy, and run your Docker images. Notably, this service will probably involve OCI Registry "(OCIR") as well to store your Docker images. Kumar's tutorial link above uses OKE
    • Oracle Developer Cloud Service
      This is a PaaS service that uses OCI resources (i.e. the servers you use for development and deployment are built in OCI).  You access this from the My Services page and not directly from OCI as shown in your screenshot. I don't recall if trial accounts include Developer Cloud Services but, if so, you can reach that by selecting My Services from the OCI menu then finding Developer Cloud under the My Services page menu (top left hamburger button).
    • OCI Functions
      This one is most intriguing to me.  Functions, based on the open-source Fn project, provides you the means to deploy one or more functions independently. If this is new to you, consider that the goal here is to provide "Functions as a Service" allowing granular control over scaling, costs, and maintenance all the way down to the function level.  This is probably the quickest to deploy of the three options. I did a node "hello world" deployment in about 5 min after reading your message above. There is a bit of overhead for setup if you've never used Functions before (I had for Python work previously) but in all shouldn't take you more than 15-30 minutes the first time and 5 minutes thereafter.
  • Jon-Eric Eliker

    The CLI command to copy as shown above does not support wildcards. The underlying API also does not seem to support wildcards (i.e. the CLI is as limited as the API in most if not all cases).  See here for details:

    OCI CLI "os object copy"
    https://docs.cloud.oracle.com/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/os/object/copy.html

    OCI API "CopyObjectDetails"
    https://docs.cloud.oracle.com/iaas/api/#/en/objectstorage/20160918/datatypes/CopyObjectDetails

    That said, the CLI "object list" operation supports wildcards somewhat so you may be able to cobble together a "list" operation feeding the objects to be copied into multiple invocations of "os object copy" commands.  Here is a simple example that copies all objects that start with "test_image" in a bucket to a remote bucket:

    for o in $(oci os object list --bucket-name frombucket --prefix 'test_image' --query 'data[*].name' | jq -r '.[]')
      do oci os object copy --bucket-name frombucket --region us-ashburn-1 --destination-bucket tobucket --source-object-name "$o"
    done
    

    Here I am using the "prefix" parameter when listing objects in a bucket to get only those that start with "test_image." Then I'm using the handy "jq" command-line tool to parse the JSON output and give me a simple list of object names.  I loop that list and issue multiple copy commands.  Since the commands are queued by default (i.e. you don't have to wait for each copy to complete), the command executes quickly even with large files to be copied.

    jq - command-line JSON processor
    https://stedolan.github.io/jq/

    You could choose to be more complex with the above example perhaps using grep or awk or some similar tool to parse to resulting list of objects with regular expression matching rather than the rudimentary "prefix" support of the "list" command alone. For example, something like this would give you a list of all files in a bucket filtered to only those that end with ".dmp"

    oci os object list --bucket-name frombucket --query 'data[*].name' | jq -r '.[]' | grep '^.*\.dmp'
    

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Once thing that every user can do is view their own user profile.  Therefore you should find that this API call works for all authenticated users even without any explicit policies in effect:

    https://identity.us-ashburn-1.oraclecloud.com/20160918/users/YOUR_USER_OCID_HERE
    

    Find documentation for this method here: https://docs.cloud.oracle.com/iaas/api/#/en/identity/20160918/User/GetUser

    Certainly, change out the API endpoint to the region of your preference.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Hi Enrique.

    I'll answer first that the Master node(s) are isolated from you and entirely managed by Oracle.  Also note that you have automatic redundancy provided by Oracle (i.e. they are running multiple masters on your behalf) plus note that you are not charged OCPU (Compute) costs for the Master nodes. I believe the implication with Oracle controlling the Master is also that you cannot readily start/stop the service itself. You'll find this explained somewhat here:

    https://cloud.oracle.com/en_US/containers/faq

    Regarding the pods "running" when the Worker nodes are stopped: I have seen the same behavior you describe and was (am) puzzled. However, if you review this documentation you'll see that "RUNNING" isn't necessarily what we assume it means (i.e. all is well and ready to be used):

    https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/

    Notably, consider this from that article above:

    The phase of a Pod is a simple, high-level summary of where the Pod is in its lifecycle. The phase is not intended to be a comprehensive rollup of observations of Container or Pod state, nor is it intended to be a comprehensive state machine.

    Instead, you may want to examine the "ready" state of the Pods.  When I do this while my Worker nodes are shut down I see "false."

    $ kubectl get pods -o=jsonpath='{range .items[*]}{.metadata.name}{"\tReady="}{.status.conditions[?(@.type=="Ready")].status}{"\n"}'
    nginx-demo-58dd5c8c87-67ns8    Ready=False
    nginx-demo-58dd5c8c87-gbjmj    Ready=False
    nginx-demo-58dd5c8c87-xm65t    Ready=False
    

    If I start at least one Worker node then I find the state changes to "true"

    $ kubectl get pods -o=jsonpath='{range .items[*]}{.metadata.name}{"\tReady="}{.status.conditions[?(@.type=="Ready")].status}{"\n"}'
    nginx-demo-58dd5c8c87-67ns8    Ready=True
    nginx-demo-58dd5c8c87-gbjmj    Ready=True
    nginx-demo-58dd5c8c87-xm65t    Ready=True
    

    Note I am using jsonpath to pull out just the data I want. You can see the full state of your Pods with either of these commands:

    $ kubectl get pods -o json
    $ kubectl describe pods
    

    I hope this helps.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    Certainly the core networking services are most highly used. While that might see obvious, consider that some may opt for OCI to support Database services (and the core networking services that it requires) even before using Compute extensively. Under this assumption, one could establish a Database Service instance attached via FastConnect (or IPSec) as a first step in OCI then pursue ad-hoc Compute systems as they build out capabilities.

    Depending on the complexity of the environment you may find that IAM services such as Compartments and Policies get a lot of use. I say this only because some new to OCI may have opted for very simple structures and limited admins so they are not fully exploring the power of IAM. Rest assured, that a sophisticated environment  can be reliably managed with a well-designed IAM implementation.

    Otherwise I'd put Core Networking/Load Balancing/Database Service/Compute high on the list of "most used." Then, in support of Database and Compute backups I think Object Storage deserves to go next followed by Block Storage (Compute volumes and DBS VM storage) then File Storage Service (for shared files). Beyond that we're seeing many more exploring Kubernetes (OKE) and the Registry (OCIR) as they consider alternatives to their monolithic, multi-server applications. For cloud-native deployments, Autonomous Database has been very popular. I expect this to grow as legacy applications gain support.

    I'm a big fan of the DNS Services but, because so many of our customers are engrained in existing DNS (often third-party hosted) that it's hard to move away immediately.  There are definitely some great features there including performance and, most recently, traffic shaping that make OCI DNS a great choice.

    The same is true about the Web Application Firewall (WAF) Service.  While many customers already have a similar feature now through deployed hardware/virtual appliances, WAF as a Service is a great option (and competitively priced) so that a number of our customers are starting to explore.

    Autoscaling is nice as well and will get even nicer once scaling up (more CPUs per system) is added along with the existing ability to scale out. Hopefully this is something that we'll see in the not-so-distant future!

    I'm looking forward to comment here from others as well. I'm interested in reading what others have found to be the most enticing or simply well-used features in OCI.

    Jon-Eric
    Mythics, Inc.

  • Jon-Eric Eliker

    There may be some third-party tools that provide this end-to-end capability but I cannot say for certain. Otherwise let me describer what I have done to achieve what you as asking. I have used the Python SDK for OCI read the existing cluster information and write the relevant Terraform files.

    For a simple example using this method to create a compartment, consider this...

    import oci
    
    config = oci.config.from_file('~/.oci/config')
    iam_client = oci.identity.IdentityClient(config)
    comp = iam_client.get_compartment(compartment_id = "YOUR_COMP_ID").data
    with open("./comp.tf", "w") as f:
      f.write('resource "oci_identity_compartment" "comp1" {\n')
      f.write('  compartment_id = "{0}"\n'.format(comp.compartment_id)) #remember this is container of the compartment not compartment itself
      f.write('  description = "{0}"\n'.format(comp.description))
      f.write('  name = "{0}"\n'.format(comp.name))
      #I am ignoring tags here
      f.write('}\n')
    

    With a little bit more work you can make this accept YOUR_COMP_ID as a parameter to the script so you can reuse to "reverse engineer" any compartment.  Another option would be to loop through all compartments using something like this:

    import oci
    
    config = oci.config.from_file('~/.oci/config')
    iam_client = oci.identity.IdentityClient(config)
    comps = iam_client.list_compartments(compartment_id = "YOUR_TENANCY_ID").data #note I'm ignoring pagination so only 21 will be listed
    i = 1
    for comp in comps: 
      with open("./comp.tf", "w") as f:
        f.write('resource "oci_identity_compartment" "comp{0}" {{\n'.format(str(i)))
        f.write('  compartment_id = "{0}"\n'.format(comp.compartment_id)) #remember this is container of the compartment not compartment itself
        f.write('  description = "{0}"\n'.format(comp.description))
        f.write('  name = "{0}"\n'.format(comp.name))
        f.write('}\n\n')
      i = i + 1
    

    This same process can be used to create a Terraform file based on any OCI resource including k8s clusters.

    See this page for the OCI Python SDK documentation about k8s resources ("container engine" resources in OCI terms): https://oracle-cloud-infrastructure-python-sdk.readthedocs.io/en/latest/api/container_engine.html

    See these pages for Terraform documentation on k8s in OCI: https://www.terraform.io/docs/providers/oci/r/containerengine_cluster.html and https://www.terraform.io/docs/providers/oci/r/containerengine_node_pool.html

    Jon-Eric
    Mythics, Inc.