Category: Uncategorized

Published on by Nayar

Elasticsearch on Docker Swarm with NGINX

On all Hosts:

sudo sysctl -w vm.max_map_count=262144

On Host 1:

1. We initialize a docker swarm. Add `–advertise-addr X.X.X.X` if inside a private network

# docker swarm init

1. We create a network on docker

# docker network create --driver overlay --subnet 10.0.10.0/24   --opt encrypted elastics

“Overlay networks connect multiple Docker daemons together and enable swarm services to communicate with each other.” [2]

2. We initialize the docker containers with 3 copies

docker service create --name elasticsearch --network=elastics \
  --replicas 3 \
  --env SERVICE_NAME=elasticsearch \
  --env "ES_JAVA_OPTS=-Xms256m -Xmx256m -XX:-AssumeMP" \
  --publish 9200:9200 \
  --publish 9300:9300 \
  youngbe/docker-swarm-elasticsearch:5.5.0

3. We get the command to generate the joining link

# docker swarm join-token worker
To add a worker to this swarm, run the following command:

    docker swarm join \
    --token SWMTKN-TOKEN \
    X.X.X.X:2377

On Worker Host
1. Type the command from the last step from host 1

# docker swarm join \
    --token TOKEN \
    X.X.X.X:2377

On Master 1

1. We now setup nginx

docker service create --name meranginx --network=elastics  nginx
docker service create --name nginx --network=elastics --mount type=bind,source=/root/meradockernginx/elasticsearch.conf,destination=/etc/nginx/conf.d/elasticsearch.conf nginx

To be continued…
#TODO: make a conf file for nginx which listens on port 9200 and uses `elasticsearch` as backend server

References:

[1] https://github.com/imyoungyang/docker-swarm-elasticsearch
[2] https://docs.docker.com/network/#network-drivers

Published on by Nayar

Duplicate Monit IDs in MMonit

when you’re using MMonit software with multiple VMs cloned from a template with monit installed, there are sometimes 2 VM get the same monit IDs. You’ll notice that there are errors on your MMonit dashboard which disappears after a while.

To view the monit id of your VMs, type the following command on your terminal

# monit -i

What do you do if you have hundreds or thousands of VMs? How will you know which ones have duplicate IDs?

I implemented a solution using SQL Triggers.

CREATE TABLE `duplicate_monitids` (
`ipaddrin` varchar(255) NOT NULL DEFAULT ”,
`monitid` varchar(255) DEFAULT NULL,
PRIMARY KEY (`ipaddrin`)
)

delimiter //
CREATE TRIGGER duplicate_monitids AFTER UPDATE
ON host
FOR EACH ROW
BEGIN
INSERT INTO duplicate_monitids(ipaddrin,monitid) VALUES(NEW.ipaddrin,NEW.monitid);
END//
delimiter ;

Then to view the VMs which have duplicate IDs, run the following SQL Query

select ipaddrin from duplicate_monitids where monitid IN (select monitid from duplicate_monitids group by monitid having count(*) > 1);

Published on by Nayar

Playing with Microsoft’s Sharepoint REST API

I couldn’t get Sharepoint to be loaded as an External Storage in Nextcloud. I had find a way to use the old school `curl` to debug the problem.

To properly authenticate, you will have to add some extra parameters to curl

$ curl –ntlm –negotiate -u <Domain>/<Username>:<Password> “http://<url>/<site>/_api/Web/getfolderbyserverrelativeurl(‘%2F<site>/<Folder>’)?$select=Length,TimeLastModified” -v

Notice the `–ntlm –negotiate`. These parameters enable “NTLM Authentication Scheme for HTTP” rather that the usual HTTP Basic Auth. This in turns make lots of back and forth between the curl and MS Sharepoint.

The logs of my HAProxy looks like this. We can see that the curl request has made 3 HTTP requests.

web sharepoint/sharepoint 0/0/1/8/9 401 503 - - ---- 2/2/0/1/0 0/0 "GET /my/_api/Web/getfolderbyserverrelativeurl('%2Fmy/MeraDocs')?=Length,TimeLastModified HTTP/1.1"
web sharepoint/sharepoint 0/0/0/8/8 401 830 - - ---- 2/2/0/1/0 0/0 "GET /my/_api/Web/getfolderbyserverrelativeurl('%2Fmy/MeraDocs')?=Length,TimeLastModified HTTP/1.1"
web sharepoint/sharepoint 0/0/0/15/18 200 3314 - - ---- 2/2/0/1/0 0/0 "GET /my/_api/Web/getfolderbyserverrelativeurl('%2Fmy/MeraDocs')?=Length,TimeLastModified HTTP/1.1"

However I was getting a 404 response in the third response initially. Turns out the API endpoints are different for OneDrive, SharePoint Online and SharePoint Server 2016:

Since I was using SharePoint Server locally, I chose the 2nd endpoint and it worked perfect.

[1] http://www.commandlinefu.com/commands/view/7005/get-a-file-from-sharepoint-with-curl

[2] https://stackoverflow.com/questions/15697157/using-curl-with-ntlm-auth-to-make-a-post-is-failing

[3] https://docs.microsoft.com/en-us/onedrive/developer/rest-api/getting-started/sharepoint-server-2016

Published on by Nayar

Kanasuk sugar

Brown sugar and white sugar are sold at the same price at the supermarket. Do you know why? Because the brown sugar is just white sugar with some added molasses to make it slightly reddish.

I saw Kanasuk sugar which was priced way above the usual sugars. Decided to give it a try.

When you open the packet, you have the pungent smell of sugarcane. It is real unrefined sugar. It tastes like sugarcane too. I used it to make my coffee only.

https://www.facebook.com/1844656215863308/videos/1937104419951820/