15 Mar 11 Zentific Datacenter: ab incunabulis

Well its been a very quiet few months and we haven’t really been keeping anyone up to date on what we are doing. Most of what is being done involves some tedious work, time off from our jobs, and a whole lot of coffee!

To that end, we have some exciting developments that took place this past weekend. Zentific has begun populating our data center! With several blade systems, 30TB of storage and blazing fast backbone switching, the Zentific data center infancy has begun. It will probably take us a few weeks to get everything configured, installed, and ready to accept production systems and not too far behind is more updates to the Zentific software.

XCP integration is on its way and that implies Citrix XenServer as well. We know that it has taken us quite some time to get these platforms ready and out the door, however, we have seen the shifts in the market and think that it is getting time to insert ourselves. We have seen the good, the bad, and the horrible as it pertains to virtualization management software. Zentific hopes to rise above those issues and give our users what they really have been looking for in a management product.

Remember, your feedback always helps us!

Also, please inquire about hosting VM’s or other infrastructure with us. Just e-mail contact@zentific.com

29 Nov 10 ZCLI again

In other news, zcli is so secure that not even valgrind can run it.
vg2 src/bin/zcli -v groups -lv
==30839== Memcheck, a memory error detector
==30839== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==30839== Using Valgrind-3.7.0.SVN and LibVEX; rerun with -h for copyright info
==30839== Command: src/bin/zcli -v groups -lv
==30839==
DBG<30839>:zcli zcli_eet.c:158 zcli_eet_setup() ~2955bit cipher generated from environment
CRI<30839>:zcli zcli_eet.c:35 _zcli_eet_setup_read() Cipher does not match session file!
CRI<30839>:zcli zcli_eet.c:36 _zcli_eet_setup_read() Stop trying to steal accounts!!!
CRI<30839>:zcli zcli_eet.c:37 _zcli_eet_setup_read() File deleted to ensure security
==30839==
==30839== HEAP SUMMARY:
==30839== in use at exit: 86,991 bytes in 1,448 blocks
==30839== total heap usage: 26,724 allocs, 25,276 frees, 14,606,294 bytes allocated
==30839==
==30839== LEAK SUMMARY:
==30839== definitely lost: 0 bytes in 0 blocks
==30839== indirectly lost: 0 bytes in 0 blocks
==30839== possibly lost: 0 bytes in 0 blocks
==30839== still reachable: 41,322 bytes in 1,379 blocks
==30839== suppressed: 45,669 bytes in 69 blocks
==30839== Reachable blocks (those to which a pointer was found) are not shown.
==30839== To see them, rerun with: --leak-check=full --show-reachable=yes
==30839==
==30839== For counts of detected and suppressed errors, rerun with: -v
==30839== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

24 Nov 10 Sneak Peek: ZCLI (Zentific Command Line Interface)

I lost my bookmark to this on my speed dial somehow and forgot about it for a few months. Now it’s time to play ketchup.

As many of you know from my post in August (yes, it was really that long ago), I have been working on things which will be used in the next version of zrpc, zpoll, zenstatd, etc. It’s undergone a bit of a name change (before: libzer now: azy), but as of a few weeks ago I have determined that it is stable enough for general client-side use. Server functionality works as well, though I have not tested it nearly as thoroughly yet. This will be a full replacement for the current libzxr, and it is smaller, faster, more flexible, and quite an improvement.

Azy, like its predecessor, implements both xmlrpc and jsonrpc. It will eventually be appearing in Zentific 2.0 (we like to think ahead). Azy support http cookies natively and, most importantly, http pipelining! This means that clients are able to connect to a server, send out however many calls they have without disconnecting, and then receive the responses asynchronously, greatly reducing rpc transmission time since there are no sockets needing to be connected between calls.

While Azy is not being used in zrpc/zpoll/etc immediately, it is making my work of coding the CLI a breeze. Essentially I just have to correctly parse the command line options/arguments, and then throw an azy-generated function into my main connection loop. The calls return very quickly, optionally with some very nice looking info thanks to eina’s logging capabilities.

Let’s talk a little about CLI security. I’ve been very conscious of sysadmin paranoia in my development; account credentials can be securely stored (using secret cipher encryption that is unique to each user and generated from the runtime environment for now, but perhaps with priv/pub key pair support later if people demand it as a feature) in zentific’s binary session file if the user desires, though the default behavior does not save any login data. Session data is stored between calls in the same way to avoid needing to log in every time an action is performed, and also to ensure that no other user can “steal” your session from the session file. Attempts to reuse a session file by another user or on another computer will result in zcli automatically detecting this and deleting the file to prevent security breaches.

Now let’s check out some direct output:

Usage: zcli [command [options]]

Command Line Interface for Zentific
Commands:
login
ls
passwd
pause
poweroff
reboot
shutdown
unpause
useradd

Options:
-v, –verbose Print runtime logs
-q, –quiet Print only requested info, no formatting
-V, –version show program version.
-R, –copyright show copyright.
-L, –license show license.
-h, –help show this message.

As you can see, I’ve tried to keep the commands identical to what a sysadmin would be familiar with to make use a breeze.
Don’t worry about the “short” command list; the list shown here is not complete, and the commands all have options that can be specified just like the regular versions. Let’s check a couple of them out!

Usage: zcli ls [options] [node[/vm]]

Command Line Interface for Zentific
ls – List
zcli ls 44444444-3333-2222-1111-000000000000
zcli ls -l 44444444-3333-2222-1111-000000000000/44444444-3333-2222-1111-123456789012
zcli ls -ld 44444444-3333-2222-1111-000000000000

Options:
-l Use a long listing format
-d List entries instead of their contents
-h, –help show this message.

This is the ‘ls’ command of zcli. As can be seen, it has a number of different uses.

The first example will output a list of all vm uuids on node with uuid ’44444444-3333-2222-1111-000000000000′, assuming it exists (VM.getAllByNode).

The second example will output the full stats of vm with uuid ’44444444-3333-2222-1111-123456789012′ on node ’44444444-3333-2222-1111-000000000000′, allowing the use of the ls command just as one would expect (nodes are directories, vms are files). Note that if ‘-l’ had not been specified, zcli would confirm or deny the existence of that vm on the specified node.

The third example will output all stats for node with uuid ’44444444-3333-2222-1111-000000000000′, as you would expect since both ‘-l’ and ‘-d’ are specified, just like with regular ls.

Usage: zcli useradd [options] username [email]


Usage: zcli useradd [options] username [email]

Command Line Interface for Zentific
useradd – Add a new user
zcli useradd -p=password username user@domain.com
zcli useradd -g=node -G=dom0_nodes -G=my_nodes
-p=supersekrit_passc0d3 node32
zcli useradd -n joe

Options:
-n, –no-disable don’t disable account upon creation (default behavior)
-D, –defaults don’t create a user, add/change default settings;
call with no other args to view current defaults
-c=GIVEN-NAME, –given-name=GIVEN-NAME
user’s given name (same as username if not specified)
Type: STR.
-g=GROUP, –group=GROUP
primary group to add user to:
(super, maint, admin, radmin, none, node OR 0-5)
Type: STR.
-l=LANGUAGE, –language=LANGUAGE
set language locale for user (eg. en_US, en_GB, fr_FR,
pt_BR)
Type: STR.
-G=GROUPS, –groups=GROUPS
secondary groups to add user to
(can be specified repeatedly)
-p=PASSWORD, –password=PASSWORD
password for user
Type: STR.
-h, –help show this message.

Here you see the very familiar ‘useradd’ command (User.addUser), which functions just like its *nix counterpart. You will notice that I have retained the ‘-D’ functionality for default settings. These can be set as expected with the options here, and they will also be stored securely in the zentific session file for reuse, used to fill in the blanks for unspecified user parameters any time a new user is created.

Now no blog post of this length would be complete without some real world test output (no video since it’s just a boring CLI!). I will demonstrate the output, with verbose setting enabled, of a very simple session:

1:14AM zmike@darc# src/bin/zcli -v login admin
Password (echo is disabled):
INF<13101>:zcli zcli_login.c:44 zcli_login() Login successful for session d9e617fd-a61b-4a2b-a7ff-07ea94522fda!
INF<13101>:zcli zcli_main.c:23 _disconnected() Disconnected
INF<13101>:zcli zcli_eet.c:209 zcli_eet_setup_write() Wrote session to disk with unix timestamp 1290579288
1:14AM zmike@darc# src/bin/zcli -v ls
INF<13102>:zcli zcli_eet.c:53 _zcli_eet_setup_read() Found session with timestamp from 6.35 seconds ago
INF<13102>:zcli zcli_eet.c:64 _zcli_eet_setup_read() Keeping old session id d9e617fd-a61b-4a2b-a7ff-07ea94522fda
INF<13102>:zcli zcli_ls.c:97 zcli_ls() 2 nodes found:
7f822640-47fd-45de-ad48-abe08b8e95fb
320563f5-2275-46e6-b172-95d2ae019f28
INF<13102>:zcli zcli_main.c:23 _disconnected() Disconnected
INF<13102>:zcli zcli_eet.c:209 zcli_eet_setup_write() Wrote session to disk with unix timestamp 1290579294
1:14AM zmike@darc# src/bin/zcli -v ls 7f822640-47fd-45de-ad48-abe08b8e95fb
INF<13103>:zcli zcli_eet.c:53 _zcli_eet_setup_read() Found session with timestamp from 10.00 seconds ago
INF<13103>:zcli zcli_eet.c:64 _zcli_eet_setup_read() Keeping old session id d9e617fd-a61b-4a2b-a7ff-07ea94522fda
INF<13103>:zcli zcli_ls.c:106 zcli_ls() 19 vms found for node 7f822640-47fd-45de-ad48-abe08b8e95fb:
1b7989ab-0984-43d2-9476-8b6fcdb695a0
7643e225-49ff-445b-be67-5d1c7f93d0e1
a7eec08f-c443-422a-8b1f-8c46dbae21ad
7da77ccc-0da3-4c22-bcac-d8b0f4a01840
4575fe57-110c-4275-bcde-4b464c5c0d0c
6b7e9bb7-54d0-414c-bd1b-6ac60bdbeed9
cb4925a9-5c03-4679-bdbb-27ef6e965fdf
72c55b78-1472-4ac7-ab68-3ba0e42421bb
d1d1adc1-2d91-4623-a0fb-77bafc869cf7
f624d978-fb7f-4e39-8bd7-6840730d7ac9
5382f520-8b75-4f8f-a72f-187a4f4a047b
d4e67e63-f311-4a84-ba8b-7cdb8623735c
acce7c69-3376-4b62-970f-067c2aa66150
b217f08c-1b7c-4763-aa12-04762af7e9cb
940f5eac-2690-48f9-bd19-0e383f03431a
937e6cbc-e798-4e7f-bb92-4d81d8d94be2
60d78a96-fcd1-c2a8-b7e6-76db9daeecf9
2f570292-e48b-438a-e58a-1f7d1b86eded
47e918dd-d875-a7dc-a91e-6ce6e813430b
INF<13103>:zcli zcli_main.c:23 _disconnected() Disconnected
INF<13103>:zcli zcli_eet.c:209 zcli_eet_setup_write() Wrote session to disk with unix timestamp 1290579304
1:15AM zmike@darc# src/bin/zcli -v ls 7f822640-47fd-45de-ad48-abe08b8e95fb/1b7989ab-0984-43d2-9476-8b6fcdb695a0
INF<13105>:zcli zcli_eet.c:53 _zcli_eet_setup_read() Found session with timestamp from 18.84 seconds ago
INF<13105>:zcli zcli_eet.c:64 _zcli_eet_setup_read() Keeping old session id d9e617fd-a61b-4a2b-a7ff-07ea94522fda
VM with uuid '1b7989ab-0984-43d2-9476-8b6fcdb695a0' exists on node 7f822640-47fd-45de-ad48-abe08b8e95fb
INF<13105>:zcli zcli_main.c:23 _disconnected() Disconnected
INF<13105>:zcli zcli_eet.c:209 zcli_eet_setup_write() Wrote session to disk with unix timestamp 1290579323
1:15AM zmike@darc# src/bin/zcli -v useradd -D
INF<13108>:zcli zcli_eet.c:53 _zcli_eet_setup_read() Found session with timestamp from 13.29 seconds ago
INF<13108>:zcli zcli_eet.c:64 _zcli_eet_setup_read() Keeping old session id d9e617fd-a61b-4a2b-a7ff-07ea94522fda
INF<13108>:zcli zcli_eet.c:165 zcli_eet_user_defaults_read() Reading user defaults from disk
INF<13108>:zcli zcli_getopt.c:383 zcli_getopt_parse() Current defaults:
uid: 0
username: test
givenname: default zcli givenname
email: emaaaaaail
active: 0
type: 3
language: en_US
1:15AM zmike@darc# src/bin/zcli -v logout
INF<13109>:zcli zcli_eet.c:53 _zcli_eet_setup_read() Found session with timestamp from 17.64 seconds ago
INF<13109>:zcli zcli_eet.c:64 _zcli_eet_setup_read() Keeping old session id d9e617fd-a61b-4a2b-a7ff-07ea94522fda
INF<13109>:zcli zcli_main.c:57 _connected() Cleared session file
INF<13109>:zcli zcli_login.c:66 zcli_logout() Logout succeeded!
INF<13109>:zcli zcli_main.c:23 _disconnected() Disconnected

The above output all has lots of pretty colors when you use the real thing, but I’m far too lazy to translate that into colors here so you’ll have to enjoy the content and not the glitter. That’s all for now, but feel free to ping me or post here with obscure feature requests!

02 Oct 10 zcenter Progress

Yeah yeah yeah, its been a while. We know.

Exciting things are happening though! The configurations have been done for the network gear going into the Zentific Datacenter and that means there will be servers and things to use soon. We will keep you updated as to what that will mean for our users.

On the software end, its been slow going as the development team has had some other obligations to concentrate on in the past 2 months. VMWare is getting worked into the Zentific core now for an early test version of the code. We are also working on capabilities to speed up development of other platforms. I am screwing around with Virtualbox management (all the while shaking my fist at oracle) which will be started along with Citrix XenServer when VMWare is done.

Bug reports! We need them! Please go download the alpha or contact us in the IRC channel for information on how to get it. For everyday you don’t, Russ (our fearless web designer) will include one kanomi code which involves killing a kitten.

Yeah…you will be responsible for that…

21 Aug 10 Video update

The previous link is no longer working, use this instead.
Zentific Video Walkthrough (HD)

13 Aug 10 Zentific: Alpha

We are pleased to announce that as of today, Zentific will officially be entering its initial alpha phase to gather user feedback prior to our release.. Come chat with us in #Zentific on FreeNode IRC, send mails to contact@zentific.com, or just try it out for yourself using the instructions at http://wiki.zentific.com/ !

Zentific Video Walkthrough

12 Aug 10 VMware…STAT!

VMware statistics are no longer covered in a haze of no hope. It may only require two calls to the vcenter and/or ESX box to retrieve all statistics.

Other good news, resource pools are getting a lot of support. This means functionality such as VMotion can be worked on in the very near future. I will be racing to get this stuff finished up because…yup…its that time of year again. School is starting

03 Aug 10 It’s a blog.

Not really a lot to say at this point that people already know, so I suppose I’ll just blog so that people know we’re still alive and kicking. Fewer svn commits than usual of late because there’s nothing left for me to do at this point. As a result, I’ve been hard at work on libZER, the Zentific EFL RPC library, which is what v2.0 of our zrpc database server will use. This will be a full rewrite of libzxr, which itself is a fork of libxr.
The key points I’ve been focusing on for ZER 1.0 are as follows

Integrating Eina and Ecore from the Enlightenment Foundation Libraries (EFL), a well-known set of open source libraries which are built around the ideas of maximum speed and stability.

Replacing bloated unnecessary glib with Eina: the EFL data type library. Here’s my favorite thing about the switch:
-rwxr-xr-x 1 root root 791K 2010-07-09 06:33 /usr/lib/libglib-2.0.so.0.2400.1
-rwxr-xr-x 1 root root 143K 2010-08-02 14:20 /usr/lib/libeina-ver-pre-svn-07.so.0.9.9


Replacing blocking socket io with asynchronous reads and writes using Ecore_Con

Replacing the current multi threaded server implementation with a much more efficient event-driven implementation using Ecore’s main loop. This will result in all function calls becoming asynchronous wherever possible, creating shorter response times for clients by avoiding blocking io and cpu sharing between threads.

Replacing JSON-glib, which currently only runs on some of our supported platforms, with cJSON, which will be linked directly into libZER to remove external dependencies. Based on the benchmarks I’ve done, the cJSON parser is at a minimum 10x faster.

That’s not all we’re overhauling/upgrading though! We’ve got KVM integration nearly complete, and our VMWare support is quickly becoming the most extensive of anyone outside of VMWare themselves. We’re also planning ahead for other database modules to interface with other types of databases.

Naturally our primary focus at this time is still the 1.0 release that always seems just out of reach, but since I have nothing interesting to share on that front I thought I’d give a teaser of things to come. Stay tuned!

08 Jul 10 Privs

I reimplemented most of the privilege code in zrpc today, the result is a much cleaner looking codebase that should run at least 2x faster. I have a few more priv-related things to fix up, then it’s on to buffer overflows and whatnot, followed at last by zensched rewrites.

Release in 6 days.

07 Jul 10 shellinabox is go

I’ve recently taken and hacked to bits a version of shellinabox to replace our old ajaxterm python script for text console connections in the interface. This is good, but I know nothing about ajax or web sites so I couldn’t tell you why.