Skip to content

Commit 5c53c92

Browse files
authored
Update howto-privatecloud.md
1 parent fc11541 commit 5c53c92

File tree

1 file changed

+1
-309
lines changed

1 file changed

+1
-309
lines changed

src/en/howto-privatecloud.md

Lines changed: 1 addition & 309 deletions
Original file line numberDiff line numberDiff line change
@@ -2,312 +2,4 @@ Title: Setting up private clouds with Simplestreams
22

33
# Set up a Private Cloud using Simplestreams
44

5-
When Juju bootstraps a cloud, it needs two critical pieces of information:
6-
7-
1. The uuid of the image to use when starting new compute instances.
8-
2. The URL from which to download the correct version of a tools tarball.
9-
10-
This necessary information is stored in a json metadata format called
11-
"simplestreams". For supported public cloud services such as Amazon Web
12-
Services, HP Cloud, Azure, etc, no action is required by the end user. However,
13-
those setting up a private cloud, or who want to change how things work (eg use
14-
a different Ubuntu image), can create their own metadata, after understanding a
15-
bit about how it works.
16-
17-
The simplestreams format is used to describe related items in a structural
18-
fashion.([See the Launchpad project lp:simplestreams for more details on
19-
implementation](https://launchpad.net/simplestreams)). Below we will discuss how
20-
Juju determines which metadata to use, and how to create your own images and
21-
tools and have Juju use them instead of the defaults.
22-
23-
## Basic Workflow
24-
25-
Whether images or tools, Juju uses a search path to try and find suitable
26-
metadata. The path components (in order of lookup) are:
27-
28-
1. User supplied location (specified by tools-metadata-url or image-metadata-url config settings).
29-
2. The environment's cloud storage.
30-
3. Provider specific locations (eg keystone endpoint if on Openstack).
31-
4. A web location with metadata for supported public clouds
32-
(https://streams.canonical.com for tools and http://cloud-images.ubuntu.com for images).
33-
34-
Metadata may be inline signed, or unsigned. We indicate a metadata file is
35-
signed by using the '.sjson' extension. Each location in the path is first
36-
searched for signed metadata, and if none is found, unsigned metadata is
37-
attempted before moving onto the next path location.
38-
39-
Juju ships with public keys used to validate the integrity of image and tools
40-
metadata obtained from http://cloud-images.ubuntu.com and https://streams.canonical.com.
41-
So out of the box, Juju will "Just Work" with any supported public cloud, using
42-
signed metadata. Setting up metadata for a private (eg Openstack) cloud requires
43-
metadata to be generated using tools which ship with Juju.
44-
45-
## Image Metadata Contents
46-
47-
Image metadata uses a simplestreams content type of "image-ids". The product id
48-
is formed as follows:
49-
50-
com.ubuntu.cloud:server:<series_version>:<arch>
51-
52-
For Example:
53-
54-
com.ubuntu.cloud:server:14.04:amd64
55-
56-
Non-released images (eg beta, daily etc) have product ids like:
57-
58-
com.ubuntu.cloud.daily:server:13.10:amd64
59-
60-
The metadata index and product files are required to be in the following
61-
directory tree (relative to the URL associated with each path component):
62-
63-
```no-highlight
64-
<path_url>
65-
|-streams
66-
|-v1
67-
|-index.(s)json
68-
|-product-foo.(s)json
69-
|-product-bar.(s)json
70-
```
71-
72-
The index file must be called "index.(s)json" (sjson for signed). The various
73-
product files are named according to the Path values contained in the index
74-
file.
75-
76-
# Tools Metadata Contents
77-
78-
Tools metadata uses a simplestreams content type of "content-download". The
79-
product id is formed as follows:
80-
81-
"com.ubuntu.juju:<series_version>:<arch>"
82-
83-
For example:
84-
85-
"com.ubuntu.juju:12.04:amd64"
86-
87-
The metadata index and product files are required to be in the following
88-
directory tree (relative to the URL associated with each path component). In
89-
addition, tools tarballs which Juju needs to download are also expected.
90-
91-
```no-highlight
92-
|-streams
93-
| |-v1
94-
| |-index.(s)json
95-
| |-product-foo.(s)json
96-
| |-product-bar.(s)json
97-
|
98-
|-releases
99-
|-tools-abc.tar.gz
100-
|-tools-def.tar.gz
101-
|-tools-xyz.tar.gz
102-
```
103-
104-
The index file must be called "index.(s)json" (sjson for signed). The product
105-
file and tools tarball name(s) match whatever is in the index/product files.
106-
107-
108-
## Configuration
109-
110-
For supported public clouds, no extra configuration is required; things work
111-
out-of-the-box. However, for testing purposes, or for non-supported cloud
112-
deployments, Juju needs to know where to find the tools and which image to run.
113-
Even for supported public clouds where all required metadata is available, the
114-
user can put their own metadata in the search path to override what is provided
115-
by the cloud.
116-
117-
### User specified URLs
118-
119-
These are initially specified in the environments.yaml file (and then
120-
subsequently copied to the jenv file when the environment is bootstrapped). For
121-
images, use "image-metadata-url"; for tools, use "tools-metadata-url". The URLs
122-
can point to a world readable container/bucket in the cloud, an address served
123-
by a http server, or even a shared directory which is accessible by all node
124-
instances running in the cloud.
125-
126-
Assume an Apache http server with base URL `https://juju-metadata`, providing
127-
access to information at `<base>/images` and `<base>/tools`. The Juju
128-
environments yaml file could have the following entries (one or both):
129-
130-
```yaml
131-
tools-metadata-url: https://juju-metadata/tools
132-
image-metadata-url: https://juju-metadata/images
133-
```
134-
135-
The required files in each location is as per the directory layout described
136-
earlier. For a shared directory, use a URL of the form "file:///sharedpath".
137-
138-
### Cloud storage
139-
140-
If no matching metadata is found in the user specified URL, environment's cloud
141-
storage is searched. No user configuration is required here - all Juju
142-
environments are set up with cloud storage which is used to store state
143-
information, charms etc. Cloud storage setup is provider dependent; for Amazon
144-
and Openstack clouds, the storage is defined by the "control-bucket" value, for
145-
Azure, the "storage-account-name" value is relevant.
146-
147-
The (optional) directory structure inside the cloud storage is as follows:
148-
149-
```no-highlight
150-
|-tools
151-
| |-streams
152-
| |-v1
153-
| |-releases
154-
|
155-
|-images
156-
|-streams
157-
|-v1
158-
```
159-
160-
Of course, if only custom image metadata is required, the tools directory will
161-
not be required, and vice versa.
162-
163-
Note that if juju bootstrap is run with the `--upload-tools` option, the tools
164-
and metadata are placed according to the above structure. That's why the tools
165-
are then available for Juju to use.
166-
167-
168-
### Provider specific storage
169-
170-
Providers may allow additional locations to search for metadata and tools. For
171-
OpenStack, Keystone endpoints may be created by the cloud administrator. These
172-
are defined as follows:
173-
174-
**juju-tools**
175-
The `<path_url>` value as described above in Tools Metadata Contents
176-
177-
**product-streams**
178-
The `<path_url>` value as described above in Image Metadata Contents
179-
180-
Other providers may similarly be able to specify locations, though the
181-
implementation will vary.
182-
183-
### Central web location
184-
([https://streams.canonical.com](https://streams.canonical.com))
185-
186-
This is the default location used to search for tools metadata and is
187-
used if no matches are found earlier in any of the above locations. No user
188-
configuration is required.
189-
190-
([http://cloud-images.ubuntu.com](http://cloud-images.ubuntu.com))
191-
192-
This is the default location used to search for image metadata and is
193-
used if no matches are found earlier in any of the above locations. No user
194-
configuration is required.
195-
196-
197-
# Deploying Private Clouds
198-
199-
There are two main issues when deploying a private cloud:
200-
201-
1. Image ids will be specific to the cloud.
202-
2. Often, outside internet access is blocked
203-
204-
Issue 1 means that image id metadata needs to be generated and made available.
205-
206-
Issue 2 means that tools need to be mirrored locally to make them accessible.
207-
208-
Juju tools exist to help with generating and validating image and tools
209-
metadata. For tools, it is often easiest to just mirror
210-
`https://streams.canonical.com/juju/tools`. However image metadata cannot be simply
211-
mirrored because the image ids are taken from the cloud storage provider, so
212-
this needs to be generated and validated using the commands described below.
213-
214-
The available Juju metadata tools can be seen by using the help command:
215-
216-
```bash
217-
juju help metadata
218-
```
219-
220-
The overall workflow is:
221-
222-
- Generate image metadata
223-
- Copy image metadata to somewhere in the metadata search path
224-
- Optionally, mirror tools to somewhere in the metadata search path
225-
- Optionally, configure tools-metadata-url and/or image-metadata-url
226-
227-
### Image metadata
228-
229-
Generate image metadata using
230-
231-
```bash
232-
juju metadata generate-image -d <metadata_dir>
233-
```
234-
235-
As a minimum, the above command needs to know the image id to use and a
236-
directory in which to write the files.
237-
238-
Other required parameters like region, series, architecture etc. are taken from
239-
the current Juju environment (or an environment specified with the -e option).
240-
These parameters can also be overridden on the command line.
241-
242-
The image metadata command can be run multiple times with different regions,
243-
series, architecture, and it will keep adding to the metadata files. Once all
244-
required image ids have been added, the index and product json files can be
245-
uploaded to a location in the Juju metadata search path. As per the
246-
Configuration section, this may be somewhere specified by the `image-metadata-
247-
url` setting or the cloud's storage etc.
248-
249-
Examples:
250-
251-
1. image-metadata-url
252-
253-
- upload contents of to `http://somelocation`
254-
- set image-metadata-url to `http://somelocation/images`
255-
256-
2. Cloud storage
257-
258-
- Upload contents of directly to environment's cloud storage
259-
- Use the validation command to ensure an image id can be discovered for a given scenario (region series, arch):
260-
261-
```bash
262-
juju metadata validate-images
263-
```
264-
265-
If run without parameters, the validation command will take all required details
266-
from the current Juju environment (or as specified by -e) and output the image
267-
id it would use to spin up an instance. Alternatively, series, region,
268-
architecture etc. can be specified on the command line to override the values in
269-
the environment config.
270-
271-
272-
### Tools metadata
273-
274-
Generally, tools and related metadata are mirrored from
275-
`https://streams.canonical.com/juju/tools`. However, it is possible to manually
276-
generate metadata for a custom built tools tarball.
277-
278-
First, create a tarball of the relevant tools and place in a directory
279-
structured like this:
280-
281-
<tools_dir>/tools/releases/
282-
283-
Now generate relevant metadata for the tools by running the command:
284-
285-
```bash
286-
juju metadata generate-tools -d <tools_dir>
287-
```
288-
289-
Finally, the contents of `<tools_dir>` can be uploaded to a location in the Juju metadata
290-
search path. As per the Configuration section, this may be somewhere specified
291-
by the tools-metadata-url setting or the cloud's storage path settings etc.
292-
293-
Examples:
294-
295-
1. tools-metadata-url
296-
297-
- upload contents of `<tools_dir>` to `http://somelocation`
298-
- set tools-metadata-url to `http://somelocation/tools`
299-
300-
2. Cloud storage
301-
302-
- upload contents of `<tools_dir>` directly to environment's cloud storage
303-
304-
As with image metadata, the validation command is used to ensure tools are
305-
available for Juju to use:
306-
307-
```bash
308-
juju metadata validate-tools
309-
```
310-
311-
The same comments apply. Run the validation tool without parameters to use
312-
details from the Juju environment, or override values as required on the command
313-
line. See `juju help metadata validate-tools` for more details.
5+
!!! Note: This page is currently being rewritten for 2.0

0 commit comments

Comments
 (0)