Categories

Last articles

Archives

Tags

3.3.2 451 group 451 research apache APIs app store appcenter apphub Appliance appliance Appliance Template appliance template application delivery application marketplace apx aws AWS aws marketplace Azure azure bento boxes bfm business big data Billing cisco citrix cloud Cloud Computing cloud computing cloud expo cloud hybride cloud migration cloud migrations cloud onboarding cloud server images cloudplatform cloudscape CloudStack cloudstack cloudwatt collaboration container crayon custom software DEB

Share

NTT Communications talks about deploying Wordpress from UShareSoft's Marketplace

Written by Gillian Gover - 15 november 2013

NTT Communications Blog

For our Japanese readers, NTT Communications and our partner, Maximize, recently published a great tutorial on getting and deploying Wordpress from UShareSoft's Marketplace.

You can read the post here. (And if you don't speak Japanese, Google Translate does a pretty good job!

Read more NTT Communications talks about deploying Wordpress from UShareSoft's Marketplace


Classified in : News, Cloud Computing, Technology, Tutorial - Tags : none - no comments


New Silverpeas Template Now Available in UShareSoft's Marketplace

Written by Gillian Gover - 12 november 2013

Silverpeas, one of our ISV partners, has recently published a new server template into UShareSoft's Marketplace.

Read more New Silverpeas Template Now Available in UShareSoft's Marketplace


Classified in : News, Cloud Computing, Technology - Tags : uforge, marketplace, silverpeas, - no comments


Curtain's Up on UForge 3.4

Written by James Weir - 29 october 2013

We are happy to announce that UForge 3.4 release is ready! This release includes a major new service - Migration. We've also moved to RBAC and made some huge improvements to supporting Windows.

Migration (Beta)

The new Migration service (launched in Beta) allows you to deep scan a live system (whether the system is virtual, cloud or physical) and import this scan as a model (i.e. server template) into UForge. This is a game-changer in the way customers create server templates in UForge. Live systems are simply teleported into UForge and transformed into a model, allowing you to re-generate images to any other target platform (traditional migration) as well as having access to the model of the live system - i.e. all the package information, licensing etc. This allows you to retrieve information on a live system for auditing purposes, or lets you tweak and change the model (updates) prior to migrating the system. Debian and RPM-based systems are supported, with Windows coming in the near future. For more information read this article, for a demo, see this video.

Role-Based Access Control

Up until now resource access has been enforced on a user per user basis. UForge resource access management has now been overhauled. Administrators can now create roles that manage entitlements that describe access to various UForge resources. These roles can then be assigned to one or more users. UForge RBAC is handled by Apache Syncope which has been embedded into the UForge platform. This module includes identity and access management, opening the door for UForge to integrate seamlessly with IAM use cases.

Windows Improvements

Ever since the arrival of UForge 3.0, users have been able to create templates using Windows as a guest OS (not available with UForge online). With this upcoming release it will now be possible to have more control of your Windows templates, including:

  • adding boot scripts to configure the instance on first (or every) boot.
  • silent installation of 3rd party software by automatically executing the installation with parameters.
  • include Windows activation keys in the template.

Sticky Packages

UForge calculates package versions and dependencies during an image generation based on a date. This allows the user to roll-forward or roll-back operating system packages in an OS profile. The negative impact on using the mechanism is that the user couldn't specify a particular package version to be part of the generation; depending on the build date of the package, an earlier or more up to date version of the package would be chosen instead. UForge now provides a mechanism to select a particular package version, known as making a package “sticky”. This increases the flexibility the user has in choosing package versions as well as protecting against destructive upgrades. This article provides more information on package dependencies.

User Usage & Statistics

The new UForge 3.4 has been updated to allow you to get usage statistics per user. We've also put in place the necessary mechanisms to integrate a third party billing system with UForge, allowing you to bill end users based on their user of UForge Factory. In addition, you can set quotas for the number of templates, generations, and migrations a user can carry out.

 


Classified in : News - Tags : UForge Release, Migration, RBAC, Windows, Statistics, Billing, Cloud Computing - no comments


"I'm My Own Grandpa" - Avoiding Dependency Hell

Written by James Weir - 15 october 2013

"I'm My Own Grandpa" is a song that tells the story of a man who, through a series of complicated marriages, becomes his own step-grandfather. Over the years, there have been many real examples that have been published in newspapers and magazines :

dependency-tree

"There was a widow and her daughter-in-law, a man and her daughter-in-law, and man and his son. The widow married the son, and the daughter the old gentleman. The widow was therefore mother to her husband’s father, and consequently grandmother to her own husband. They had a son, to which she was a great-grandmother: now, as the son of a great-grandmother must be either a grandfather or great-uncle, this boy was one or the other" - Hood's magazine in 1884

Reading this reminded me of the confusion and pain that I have been through on the numerous occasions when I have had to install or upgrade packages on a system. A typical scenario would be, when installing a new package, I would quickly realise that this package required a whole bunch of other packages. Once those are installed, then guess what, I was missing their package dependencies as well, turning a simple installation into a nightmare. Welcome to dependency hell.

What is a Dependency

A dependency is a piece of information in a software package that describes which other packages it requires to function correctly. Many packages require the OS's system libraries as they provide common services that just about every program uses (filesystem, network, memory etc). For example, network applications typically depend on lower-level networking libraries provided by the operating system. The principle behind package dependencies is to share software, allowing software developers to write and maintain less code at a higher quality. Operating systems have thousands of packages. In the world of virtualization and cloud computing, it is becoming imperative to strip down the number of operating system packages to just the required packages to run a particular application. This process, known as JeOS (pronounced "juice") standing for "Just Enough Operating System" is a very painful manual process. So much so that many operating system vendors now supply a core operating system ISO with the minimum set of packages required to boot the system. The fun then begins as you manually install only the packages (and their dependencies) required to run your application.

Calculating Package Dependencies Automatically

One of the primary goals of the UForge platform was to avoid dependency hell when creating custom stacks (known as server or appliance templates). It was also important not to use core ISO images as the base of generating custom stacks as these are opaque, but to expose all the operating system packages in your template. Each appliance template has an os profile. This os profile contains all the operating system packages you have chosen. To help you get started, you can either choose from one of several standard os profiles (choosing the "bottom-up" approach) or use the smallest standard os profile available (choosing the "top-down" approach). You can then complement this package list by searching for other packages you require from the os distribution or upload and add any custom native packages (via "My Software").

create-os-profile

So far you have not had to worry about package dependencies. You only need to add the packages you know you require to run your application (i.e. the first level dependencies your application requires).

os-profile-packages

Package dependency checking occurs when you generate a new image. During the first phase of generation, UForge calculates automatically all the dependencies of each package in the os profile as well as any packages contained elsewhere in your custom stack (custom software in "My Software" and "Projects"). All missing packages are automatically added to your os profile. For each package added, this package's dependencies are also checked. This process continues until all the dependencies have been met. The end result is a complete dependency tree of all the packages you require to run your application. What normally may take hours or days to do manually, only takes a few minutes.

Under The Hood

Each package has meta-data on what the package requires (that is, what the package depends on) and what it provides in terms of functionality. This meta-data varies on the package type (RPM, DEB etc). For example: RPM:

  • Provides - one or more libraries or services this package provides
  • Requires - one or more packages required for this package to run correctly
  • Conflicts - one or more packages this package conflicts with (the other packages cannot be installed if this package is installed)
  • Obsoletes - this package supersedes another (can be used for package renaming)

DEB:

  • Provides - one or more libraries or services this package provides
  • Requires - one or more packages required for this package to run correctly
  • Conflicts - one or more packages this package conflicts with (the other packages cannot be installed if this package is installed)
  • Obsoletes - this package supersedes another (can be used for package renaming)
  • Recommends - one or more packages that is recommended to be installed (optional)
  • Suggests - one or more packages that is recommended to be installed (optional)

UForge works with repositories. These repositories can be official operating system mirrors or private repositories. Other repositories including EPEL can also be handled by UForge. For each package in the repository, UForge uses the meta-data to calculate the dependencies in the os profile of your appliance template. The dependency calculation is done using a specific moment in time. This date is stored in your appliance (Appliance object, attribute: lastPkgUpdate). When creating a new appliance, the "lastPkgUpdate" date is the same date as the created time. Chosen package versions and dependencies are calculated by ensuring that they are equal to or less than the "lastPkgUpdate". Let's take an example. Imagine you create a new appliance on June 17th 2013, 17:00 GMT+1, and you choose package A, B and C in the os profile. Note that packages A, B and C may have more than one version (updates added to the repository due to bug fixing and or new features). The versions displayed for A, B and C will be dates of each of these packages closest (but inferior) to the "lastPkgUpdate".

package-updates1

In this case: package A v1.3, B 7.2, and C 3.2 is chosen. All dependencies for A, B and C will be calculated based on these versions using the package meta-data existing in UForge.

Wait A Minute, What About New Updates ?

As you probably know, packages evolve as bugs are fixed and new features are added. These new packages become available in the operating system repository. UForge uses an internal mechanism to check for any new package update available in the repository, and, if found, adds the meta-data of this package to its own database. Using this process, UForge builds a history of the operating system, as it keeps references to the old packages that are being replaced by the update. These updates do not get taken into account for your current appliance when generating a new image. UForge ensures the same package versions are used regardless of when you generate your image, the same image is generated time after time. This is due to always using the "lastPkgUpdate" timestamp of the appliance in question. Ok great, but what if I actually wanted to include these updates in my next generation? Well, it's a simple matter of updating the "lastPkgUpdate" of the appliance. UForge actually provides this update information for you. It knows the "lastPkgUpdate" of each of your appliances, as well as the list of packages in your os profile and therefore calculates whether or not any new updates are available.

package-updates2

In this case, UForge will notify you that three updates are available. Note, that for package B even if there is an intermediary package (version 7.3), only the last one is taken into account. UForge provides a way for you to see the evolution of package updates in the os profile. Clicking on the "update" button for a particular template produces a time graph of all the updates for the os package list through time. This allows you decide where to set the "lastPkgUpdate" to. By changing the "lastPkgUpdate" you can run a simulation that will describe which packages will be updated giving the current and new version of each package impacted.

os-profile-updates3

By calculating package dependencies in this way, there is an interesting side-effect. Not only can you update (roll-forward) your packages, but you can generate an image from packages that were delivered in the past (roll-back). This is as simple as setting "lastPkgUpdate" to an earlier date. This is neat, as you don't necessarily need to always take the bleeding-edge latest updates of an operating system. This also goes a way forward to helping solve "destructive upgrades".

Destructive Upgrades

When you perform an upgrade of a package on a live system, the package manager will overwrite the files that are currently on your system with the files provided by the new package update. This works perfectly when you assume that the package you are replacing is backward-compatible. The problem is that it is extremely difficult to ensure this when delivering a new package version. Let's imagine that you wish to upgrade a package that depends on a newer version of php. When you upgrade, you will also upgrade the current version of php with the new one. If the php package is not completely backward-compatible then other applications that relied on the old php package may break. This is known as destructive upgrades. Once you have done a destructive upgrade, it is:

"...hard to undo, or roll back, an upgrade. Unless you or your package manager makes a backup of all the files that got replaced, you cannot easily undo an upgrade. Finally, while the package manager is busy overwriting all the files that belong to a package, your system is temporarily in an inconsistent state in which a package may or may not work properly. Hit the power switch on your computer halfway through your next OS upgrade and see if the system will still boot properly!" -- Pjotr Prins (Nix fixes dependency hell on all Linux distributions)

With the package "time machine", you can test whether you will have a destructive upgrade by simulating the upgrade, generating the image and testing it, prior to upgrading your production systems. Using UForge, you can also determine which packages will be updated and to which versions.

Sticky Fingers

Being able to roll-forward or roll-back the packages is all well and good, but what if we wanted to force a particular version of a package to be part of the generated image? Due to the current package version calculation being based on a particular date (the lastPkgUpdate) it is impossible to specify a particular package version to be part of the generation, as depending upon the build date of the package, potentially an earlier or more up to date version of the package may be chosen instead. To get around this issue, UForge provides a mechanism to enforce a particular package version. This is known as making a package "sticky".

os-profile-sticky-pkg

In the os profile, by clicking on the "sticky" button, a list of all the versions of the package is displayed, allowing you to choose the version you require. Once chosen, this package becomes sticky. During image generation, this package version is chosen regardless of the current "lastPkgUpdate". All the package dependencies of this package are also calculated.

os-profile-sticky-pkg2

A simple click on the "sticky" package will make the package to become "unsticky", and the package version will be calculated as normal. To summarize, UForge provides powerful package dependency checking to create a JeOS appliance template with little to no manual effort - helping you to avoid dependency hell and ensure you will never be your own grandpa (or sister for that matter).


Classified in : Technology, Tutorial - Tags : UForge, Appliance, Template, Package, Package Dependency, Dependencies, RPM, DEB, OS packages, OS profile, destructive upgrades, live update - no comments


Two Workloads. One Cloud.

Written by James Weir - 20 september 2013

Today, UShareSoft's partner, Citrix, is announcing CloudPlatform 4.2, which will orchestrate both cloud-native workloads and existing enterprise workloads within a single cloud management platform.

Why is that important for UShareSoft and our customers?

Read more Two Workloads. One Cloud.


Classified in : News, Cloud Computing, Technology - Tags : cloudstack, cloudplatform, apache, citrix, cloud computing, hybrid cloud, enterprise workload, cloud migration, workload migration - no comments


Rss feed of the articles

page 1 of 5 next »