# Manually Add External Credentials

If you didn't enable `External APIServer` option when creating the cluster, you'll find that there are no external credentials in cluster management, thus you cannot access the APIServer through the internet.

In this case, you can manually add external credentials to the cluster. This section will introduce detailed steps.

> **Note:** Adding external credentials requires restarting the master node's apiserver, so please operate during off-peak business hours.

## Create ULB

> **Note:** The external ULB is not created along with UK8S, which will not be deleted synchronously after the deletion of the UK8S cluster. Therefore, manual deletion of this ULB is necessary

You need to manually create an external ULB for access to the master node.

In the "Load Balance ULB" page, click "Create Load Balance", choose "request proxy type" for the load balance type, "external network" for the network mode, choose the VPC where the UK8S cluster is located for the attached VPC, and choose Elastic IP related configurations for creation, as shown below:

<!-- image-todo -->

## Create VServer

First, find the ULB used by UK8S **Internal APIServer** and note down its name. In this example, it's "uk8s-cx13ag67xdh-master-ulb4".

On the details page of the external ULB created in step 1, click on "VServer Management", then click "Create VServer", choose "TCP" protocol and 6443 port, choose service node as "Copy from other VServer", choose ULB instance as the ULB used by the internal APIServer, and fill in other creation parameters, as below:

<!-- image-todo -->

After creating, you can see the health check status of the three master nodes changed to "normal" on the VServer details service node page, as shown below:

<!-- image-todo -->

## Regenerate SSL Certificate

**Note, the following operations need to be performed on all 3 master nodes**

Note down the EIP of the external ULB you just created.



Assume the EIP is `99.99.99.99`.

Log into the master node via ssh, and install the SSL certificate tool:

```bash
curl -L -o cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
curl -L -o cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x cfssl*
sudo mv cfssl* /usr/local/bin/
```

Continue on the master node where you just installed the tool to query the current APIServer service's SSL certificate information：

```bash
openssl x509 -noout -text -in /etc/kubernetes/ssl/kube-apiserver.pem | grep DNS
```

You should see some output like this:

```
 DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster, DNS:kubernetes.default.svc.cluster.local, IP Address:127.0.0.1, IP Address:172.16.0.1, IP Address:10.23.145.160, IP Address:10.23.151.92, IP Address:10.23.76.232, IP Address:10.23.226.61
```

Note down all the addresses above.

Next, create a `kube-apiserver-csr.json` file anywhere, and add all the internal network addresses and your ULB's external network address to the file:

```json
{
    "CN": "kubernetes",
    "hosts": [
        "127.0.0.1",

        // Replace with the IP address outputted by the openssl command
        "172.16.0.1",
        "10.23.145.160",
        "10.23.151.92",
        "10.23.76.232",
        "10.23.226.61",

        // Replace with the EIP of the external ULB you just created
        "99.99.99.99",

        // Replace with the DNS outputted by the openssl command
        "kubernetes",
        "kubernetes.default",
        "kubernetes.default.svc",
        "kubernetes.default.svc.cluster",
        "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [{
        "C": "CN",
        "ST": "BeiJing",
        "L": "BeiJing",
        "O": "k8s",
        "OU": "System"
    }]
}
```

**After replacing, remember to delete the comments in the json file.**

Create another file `ca-config.json`, and add the following content：

```json
{
    "signing": {
        "default": {
            "expiry": "87600h"
        },
        "profiles": {
            "kubernetes": {
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ],
                "expiry": "87600h"
            }
        }
    }
}
```

Back up the previous SSL certificate:

```bash
cp -r /etc/kubernetes/ssl /etc/kubernetes/ssl-back
```

Run the following command to generate the certificate:

```bash
cfssl gencert -ca=/etc/kubernetes/ssl/ca.pem -ca-key=/etc/kubernetes/ssl/ca-key.pem -config=ca-config.json -profile=kubernetes kube-apiserver-csr.json | cfssljson -bare /etc/kubernetes/ssl/kube-apiserver
```

Use the following command to verify that the external IP has been added (replace `99.99.99.99` with your own external IP)：

```bash
openssl x509 -noout -text -in /etc/kubernetes/ssl/kube-apiserver.pem | grep '99.99.99.99'
```

Restart apiserver so that the certificate can take effect:

```bash
systemctl restart kube-apiserver
```

The steps to access the external network APIServer via kubectl are consistent with accessing the internal network APIServer. You can copy the "internal network credentials" on the UK8S cluster details page, modify the server field in it, and change the internal network IP address to the external IP address of the ULB, as shown below:

<!-- image-todo -->

Through this credential, you can now access the cluster from the Internet.