|
Welcome to the Argonne BG/L system
The topics covered here should cover everything that a new user who
is moderately experienced with Linux should need to know to get
started on the BG/L system. Links to other documents with more
information on specific topics are also provided.
Complete detailed documentation may be found on the
How to use BG/L pages.
| How to... |
Information |
| Get Help |
The BG/L Support Team can assist you
with your questions and requests. The best way to receive support
is to send an email to support@bgl.mcs.anl.gov,
which will log a ticket to the BG/L Problem Tracking System and forward the email to
BG/L support staff. We hope to have a web-based interface to the problem tracking system
in the future.
We hope you can find much of the information you'll need through
the BG/L web pages. The home page is
http://www.bgl.mcs.anl.gov.
From that page, you will find links to software and
other documentation, online tutorials, mailing lists
and current status information.
|
| Get an Account |
Any ANL MCS employee may request an account on the Argonne BG/L
system. In addition, members of the BGConsortium,
collaborators on certain MCS projects and many others may also
request accounts. Currently, access to the machine is limited to users
with an MCS account. If you do not have an MCS account but would
like access to the BGL machine, please send email to support@bgl.mcs.anl.gov
with details about the work you plan on doing on BGL. Once your request has been evaluated, we will send further
information explaining how to complete your account application.
IMPORTANT: If you already
have an MCS account, please use your personal MCS
account page
to request the BG/L resource instead of applying for a
new BG/L account.
|
| Connect and Log In |
Access to the BGL system requires that a SSH key be installed in your BGL account.
Instructions for setting up your ssh key were included in the 'Welcome' mail sent
when your account was created. In summary, please send your public key to
support@bgl.mcs.anl.gov.
Software that uses the ssh protocol
must be used to connect to the system for interactive sessions.
See Accessing the Argonne BG/L
system
for detailed information on using ssh and the associated scp
command.
Once you have an account and your ssh key installed, you may log onto BGL.
To log onto the BG/L system from a unix machine, you would
ssh to the cluster alias bgl.mcs.anl.gov with
the command:
ssh -l <bgl_username> bgl.mcs.anl.gov
Once you have connected, you will be prompted for the SSH passphrase
associated with the public key installed in your account on BGL.
"Accessed Denied" error messages at
login usually mean there is a password or username problem;
contact support@bgl.mcs.anl.gov for help.
The bgl.mcs.anl.gov name refers to all currently available
user login hosts. Users are placed onto the real hosts in
round-robin fashion, as a result they may end up on a different
host from one login to the next. Because the real hosts may
change without notice, we suggest that you use the alias and
not the real hostnames to log onto the system.
|
| Set Up Your Environment |
Once you are logged onto the BG/L system, you will be sitting
at a Unix shell prompt in your home ($HOME) directory. If you
have never used UNIX before, you will need to learn some basic
commands before you will be able to submit jobs. A number of
links to tutorial information may be found at the
Unix Guru Universe Beginners' Pages (external link), particularly the "Getting Started"
section.
Your account will probably be setup with the default shell
tcsh. Two other shells,
bash and
zsh are also provided.
We hope to have an automated way for you to change your default shell soon,
in the meantime, please send email to support@bgl.mcs.anl.gov with
the shell you prefer.
The BG/L system uses the MCS SoftEnv package to manage access to applications
on the machine. The first time a users log onto BG/L, a '.softenvrc'
configuration file will automatically be created in the users's
home directory. This file will be set up with the default
applications environment - typically there will be a single line
with '@default' in the file. Depending on the user's
applications, further modifications to the .softenvrc file may be
necessary.
SoftEnv man pages are available in the default environment (use
'man softenv'). The command 'softenv' will list all available
applications. Users wishing to gain a more complete understanding
of how softenv works may read the complete softenv
documentation.
|
| Store Files |
There are two primary file systems on the BG/L system available to users - the home directory
area in /home, and the PVFS filesystem mounted on /pvfs.
Your home directory, /home/<username>, is on a NFS
filesystem. This filesystem is mounted across all login servers and the I/O nodes
associated with your job. All scripts and jobs which make use of home directory
space should use the /home/<username> path (and not the /bgl/home1 mount path).
In general, there should be no need to
manually copy files out to the nodes. The home directory space is
very limited and is not accessible from the compute
nodes. The home directory space is backed up on a nightly basis.
The second file system is the PVFS scratch filesystem,
which is mounted on /pvfs. This filesystem is also mounted across all login servers
and the I/O nodes associated with your job. However, it is currently not possible to
run job executables out of /pvfs, or to use a /pvfs directory as the working directory for
a job. Also, it is not possible to use /pvfs for the job .output and .error files. Any output
to /pvfs must be handled directly by your application using absolute paths, and cannot rely
on assumptions about the job's working directory. We hope to eliminate this restriction in
the near future.
The primary purpose of PVFS is to act as a high-performance parallel scratch filesystem;
it is not intended for long term archival storage, since it is not backed up.
Jobs that perform large amounts of parallel file I/O, particuarly with larger file sizes,
are encouraged to use /pvfs. Please note that larger block sizes ( >= 64 Kbytes)
will provide the best performance on /pvfs, since this minimizes the TCP/IP overhead.
NOTE: The /pvfs file system is not backed up, so we highly recommend that
you archive important data to tape.
For information
about how to achieve the best performance and avoid the various
problems please read the
BG/L File Systems page.
|
| Compile Code and Run jobs |
Compiling your code for the BG/L system is too complex to discuss
in detail in this document. We are
writing a how-to document and a pointer will be provided from this
page and the 'Using BGL' section of the web as soon as the how-to document is ready.
A useful document to read is the file located on the BGL login servers:
/software/common/doc/BGL-Hints-Tips.txt
You might also browse the
ANL BGL Wiki for other useful information.
If you wish to try using the machine with a precompiled binary, you can use the
HelloWorld executable provided on BGL in /software/common/bin/HelloWorld.rts.
A example of one way to compile/link foo.c into the executable named foo
is provided below.
blrts_xlc -I/bgl/BlueLight/ppcfloor/bglsys/include -g -O -qarch=440 \
-qmaxmem=64000 -o foo foo.c -L/bgl/BlueLight/ppcfloor/bglsys/lib \
-lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts
Jobs are now run by submitting them to the job resource manager and scheduler.
Generally, 32 node partitions are available for use at any time except during
reservations.
Larger runs require a reservation (send mail to
support@bgl.mcs.anl.gov). Please see the
documentation in the
How to use BG/L pages.
When a user's job starts, the BG/L service node runs the job
on the compute and I/O nodes associated with the job.
Because the OS on the compute nodes is a simple microkernel, access to
compute nodes is not possible at this time.
The Running Jobs document provides
details about how run your jobs.
Please read the
BG/L Policies
page which includes information about machine usage.
|
| Manage Email |
When you get a BG/L account, a mail alias is automatically
created for you. BG/L mail server will send
any mail addressed to you to the address installed as your alias.
This alias is automatically modified whenever you change
your preferred mail address on your MCS personal account
page accessible from the
main MCS account area.
There are a number of mailing lists associated with the Argonne BG/L system.
A complete list of the mailing lists and access to their
archives may be found on BG/L Mailing Lists page.
Two important mailing lists are the bgl-users@bgl.mcs.anl.gov and
the notify@bgl.mcs.anl.gov lists.
| List |
Description |
| bgl-users@bgl.mcs.anl.gov |
Used by system administrators for long term information such
as security policies. All users on the BG/L system are members of this
mailing list.
|
| notify@bgl.mcs.anl.gov |
Used by the administrators to communicate operational
status that would be of interest to the active user
community (for example, short emergency outages).
Membership in the notify list is optional. Users are
subscribed to notify at account creation time. After that, membership on
that maillist is managed by the users with the
'notifyme' command:
| 'notifyme -y' |
| start notifications |
| 'notifyme -n' |
| stop notifications |
|
|
| Get More Help |
If, after reading the documentation, you have questions or require
assistance, please contact the
BG/L Systems Team.
|
|