Date: Tue, 26 Feb 2002 16:17:40 +0100
To: rietveld_l@ill.fr
From: Jonathan WRIGHT [wright@esrf.fr]
Subject: RE: X'Cellerator and Proportional counter

>1) Good resolution in 2theta (or equivalent measure of d*);
>
>2) A reasonably complete model that includes all the important
>structural and instrumental variables (including peak shape);
>
>3) Good counting statistics.

4) Good q-range, extending to high angles or short d-spacings, to have the
best possible structural "resolution". eg: a 1.0 angstrom structure with
CuK_alpha radiation would mean data where peaks can still be seen as far as
100 degrees in two theta. This is clearly sample dependent, but I was
slightly surprised to see that the data posted to the list had constant
counting statistics across the two theta range. At least with x-rays,
aren't people spending longer on the high angle data for Rietveld refinements?

Cheers,

Jon

From: "L. Cranswick" [lzc@dl.ac.uk]
Subject: Re: X'Cellerator and Proportional counter - VCT data
To: rietveld_l@ill.fr
Date: Tue, 26 Feb 2002 22:15:07 +0000 (GMT)

> 4) Good q-range, extending to high angles or short d-spacings, to have the
> best possible structural "resolution". eg: a 1.0 angstrom structure with
> CuK_alpha radiation would mean data where peaks can still be seen as far as
> 100 degrees in two theta. This is clearly sample dependent, but I was
> slightly surprised to see that the data posted to the list had constant
> counting statistics across the two theta range. At least with x-rays,
> aren't people spending longer on the high angle data for Rietveld refinements?
> 
> Cheers,
> 
> Jon


Which hardware vendor's software allows the collection of
variable count time data and the merging into a single
file with ESDs (or count time)?

As far as I can tell - the answer is "None" 
(some vendors may say it is possible but in practise it is so 
kludgy and cumbersome to perform that it is almost impossible 
to do this effectively).  
Thus you may have Rolls Royce quality diffraction hardware - 
yet by not having modern data collection features in the
control software - vendors would have you use their hardware 
as though you were riding a rusty bicycle.

------

However, rather than just doing ad-hoc increasing of count times, 
there is a Variable Count Time program available (VCT) 
(Fortran source and DOS executable) for collecting Madsen 
and Hill style VCT data.  This also generates a command script 
for a Philips 1710 diffraction system (which you can then send 
down the serial control cable if you have the software for 
doing this).

   http://www.ccp14.ac.uk/ccp/ccp14/ftp-mirror/csirominerals-anon-ftp/pub/xtallography/variable-count-time/

References:

"Collection and analysis of powder diffraction data with near-constant 
counting statistics", I. C. Madsen and R. J. Hill, J. Appl. Cryst. 
(1994). 27, 385-392, http://www.iucr.org/cgi-bin/paper?hz0014 

I. C. Madsen and R. J. Hill, Adv. X-ray Anal. (1992), 35, 39-47 

----

Though there seems to be source code, executable or algorithms
available based around the following abstract (?):

W. I. F. David, Accuracy in Powder Diffraction, Abstract P2.6 NIST 
Special Publication No. 846, Page 210, (1992) National Institute of 
Standards and Technology, Gaithersburg, MD, USA. 

------

Fullprof and Winplotr can read Madsen and Hill VCT data 
without any required conversion.  (format 11 in the manual)
  
For GSAS (and programs that use GSAS ESD format), there is 
a tutorial for converting Madsen and Hill Variable Count 
Time data into GSAS ESD format at:

  http://www.ccp14.ac.uk/solution/gsas/convert_vct_data_to_gsas.html

This is using Mark Bowden's VCTCONV for DOS program.
 
  http://www.ceramics.irl.cri.nz/Convert.htm

-----

If you are using the above Madsen and Hill code and have a Philips
1710 (and a way of sending the script down the serial control cable);
you can use the Philips PW 1710 script it generates as a template to 
generate "custom" Variable Count Time data collection scripts.  
This can be useful for some refinements - e.g.,  collect intense 
subcell reflections and weak supercell reflections with similar 
counting statistics.  The above webpage tutorial shows such 
an example.

As mentioned in recent talk notes (using data collected
by Jeremy Cockcroft as the example), Variable Count Time can 
also be very  effective for phase ID, quantitative analysis,
powder indexing and unit cell refinement - where you collect 
every minor to trace peak with the same counting statistics 
as the major peaks.  (VCT can allow you to refine the
structures of trace phases - assuming there is not excessive
overlap with peaks of the major phases)

Refer: (slides 17 to 29)
  http://www.ccp14.ac.uk/poster-talks/axaa2002/

Lachlan.



-- 
-----------------------
Lachlan M. D. Cranswick

  1st Feb to 8th Feb - then 18th Feb to 17th March 2002
  CSIRO Division of Minerals
  PO Box 312, Clayton South,  Melbourne, Victoria, 
  Australia, 3169.  Tel:  (613)  9545-8802
                    Fax:  (613)  9562 8919

Collaborative Computational Project No 14 (CCP14)
    for Single Crystal and Powder Diffraction
  Birkbeck University of London and Daresbury Laboratory 
Postal Address: CCP14 - School of Crystallography,
                Birkbeck College,
                Malet Street, Bloomsbury,
                WC1E 7HX, London,  UK
Tel: (+44) 020 7631 6850   Fax: (+44) 020 7631 6803
E-mail: l.m.d.cranswick@dl.ac.uk    Room: B091
WWW: http://www.ccp14.ac.uk/

From: "Shankland, K (Kenneth) " [K.Shankland@rl.ac.uk]
To: rietveld_l@ill.fr
Subject: VCT 
Date: Wed, 27 Feb 2002 10:34:47 -0000

>Which hardware vendor's software allows the collection of
>variable count time data and the merging into a single
>file with ESDs (or count time)?

It may not be able to be done 'routinely' but it's hardly
'almost impossible'.  However, it's probably true to say
that most people would use a VCT facility if it was 
an option, or better still, the default option.  Ultimately,
it all depends on how good you want your data to be.  
Most folks are simply wary of changing their tried and tested
methods and are content with the results they normally get.

*Every* data set for structure solution that we've collected
on BM16 of the ESRF has been collected in a VCT fashion, and
that's going back as far as experiments run in 1997.   

The benefits of VCT are clear for solution (both by direct
methods and global opt) and refinement.  It's so easy to
apply and to do it "correctly" takes only a little more 
effort than to say "oh, I counted for twice as long at high
angles"....

Cheers,

Kenneth

Date: Wed, 27 Feb 2002 13:15:03 +0100
To: rietveld_l@ill.fr
From: Armel Le Bail [alb@cristal.org]
Subject: Re: VCT 

>The benefits of VCT are clear for solution (both by direct
>methods and global opt) and refinement.  It's so easy to
>apply and to do it "correctly" takes only a little more
>effort than to say "oh, I counted for twice as long at high
>angles"....

Using variable slits (VS) can be considered as a (bad) way
to obtain data analogous to VCT data. However, few Rietveld
programs allow the direct use of VS data, you have to correct
them before refining the structure or extracting structure factor
amplitudes.

So, Rietveld programmers, please add the possibility to cope
directly with VCT and VS data.

And manufacturers, please make VCT measurements easy.

(one can suppose that both programmers and manufacturers
are listening, even if few are writing here ;-).

Armel

From: Reinhard Kleeberg [kleeberg@mineral.tu-freiberg.de]
Date: Wed, 27 Feb 2002 13:52:21 +0000
To: rietveld_l@ill.fr
Subject: Re: X'Cellerator and Proportional counter - VCT data

Hi,
some naive questions from a none-mathematician without experience in
structure solution:
At 22:15 26.02.02 +0000, Lachlan wrote:
>
>Which hardware vendor's software allows the collection of
>variable count time data and the merging into a single
>file with ESDs (or count times)?

What about a simple manually dividing the pattern in 2Theta ranges with
increasing counting time? This can be done with all diffractometers. Each
range contains either the pure counts or the cps and the counting time. As
I understand, the Rietveld programs should use these two types of
information to get the ESDs or store the ESD information separately in the
data file (see GSAS ESD format). As another example, BGMN derives the ESDs
from the range headers to use this in its weighting method. Therefore, it
is not recommended to use data with wrong statistic information (say, wrong
counting time entries in converted files) or to falsify the statistics by
background substraction (as Joerg Bergmann told me).
In extreme cases, one could define manually a lot of measuring ranges (of
course, a lot of work, but for structure refinement ...).

Has anyone experience in using data with pre-selected counts (variable
counting time for each step, until the pre-selected number of counts is
reached)? Than the counting statistic is equal for each step. This can
easily done by the most diffractometers, but I know nothing about the
reaction of Rietveld programs (using different weigthing methods) if they
have to use such data (and whether the programs are able to identify these
data type automatically).
On the other hand, for low background configuration/samples, such a
measurement will need a lot of time, and it will seldom be practicable.

Reinhard
Dr. R. Kleeberg
TU Bergakademie Freiberg
Institut für Mineralogie
Brennhausgasse 14
D-09596 Freiberg
Germany
Tel. +49 (0) 3731-39-3244
Fax. +49 (0) 3731-39-3129

Date: Wed, 27 Feb 2002 15:19:11 +0100
To: rietveld_l@ill.fr
From: Jonathan WRIGHT [wright@esrf.fr]
Subject: Re: VCT 

>So, Rietveld programmers, please add the possibility to cope
>directly with VCT and VS data.

For the VCT data at least, don't we just take
counts=total_counts/total_time and esd=sqrt(total_counts+k)/total_time,
where k=0, 0.5 or 1 according to taste, and then use an esd data format for
the refinement? There doesn't seem to be anything a Rietveld programmer can
do to help. Are there *any* modern Rietveld programs which don't support an
esd data format and conventional least squares weights??

A nice extension might be to get the Rietveld program to write out the
derivatives of various parameters of interest for each point in the pattern
so you can work out which particular data points are most important for the
refinement of each parameter. Then send the diffractometer off to
concentrate on the points where the derivatives are largest. That kind of
requires that you can solve the problem before you collect the data, or at
least much faster than it takes to collect the full dataset, so I doubt if
it'll take off ;-) 

>And manufacturers, please make VCT measurements easy.

Given that most (all?) already let you program a series of scans there
doesn't seem to be a great deal the manufacturers can do to help either.
All that is needed is a trivial program to combine the data. I guess this
doesn't exist because of the infinite variety of file formats it'd end up
needing to support and possibly that no one would try to write up something
you can do with excel.

Cheers,

Jon

Date: Wed, 27 Feb 2002 16:04:28 +0100
To: rietveld_l@ill.fr
From: Armel Le Bail [alb@cristal.org]
Subject: Re: VCT 

>For the VCT data at least, don't we just take
>counts=total_counts/total_time and esd=sqrt(total_counts+k)/total_time,
>where k=0, 0.5 or 1 according to taste, and then use an esd data format for
>the refinement? There doesn't seem to be anything a Rietveld programmer can
>do to help. Are there *any* modern Rietveld programs which don't support an
>esd data format and conventional least squares weights??

Sure, obviously, no help is needed from a Rietveld programmer if you
transform your original data back into the classical form (just
like if there was no VCT, no VS).

But you may consider using the case where counts = counts ;-)
In that case, VCT or VS data would allow to give more weight
than usually at increasing diffracting angle, in a way that mimics
neutron data.

> >And manufacturers, please make VCT measurements easy.
>
>Given that most (all?) already let you program a series of scans there
>doesn't seem to be a great deal the manufacturers can do to help either.

Again, the idea is to use the data as they were registered (counts=counts)
Rather than to program a series of scans (quite tedious) which would
need to have several scales in your refinements (hypothesis counts=counts),
you may consider that the manufacturer allows you to select (at a push
of one button) between a few possibilities of simple laws giving counting
times as a function of diffracting angles, and that the Rietveld program would
also have these laws inside.

So let me ask again (more clearly ?) :

Rietveld programmers, please add the possibility to cope
directly with VCT and VS data.

And manufacturers, please make VCT measurements easy.

And let me add : both of you, work together coherently.

Armel

PS - Anyway, you can continue to use Excel if you want.

From: "Joerg Bergmann" [bergmann@rcs1.urz.tu-dresden.de]
To: "rietveld_l@ill.fr" [rietveld_l@ill.fr]
Date: Wed, 27 Feb 2002 16:26:12 +0100 (CET)
Subject: Re: VCT 

Dear Jon,

On Wed, 27 Feb 2002 15:19:11 +0100, Jonathan WRIGHT wrote:

>A nice extension might be to get the Rietveld program to write out the
>derivatives of various parameters of interest for each point in the pattern
>so you can work out which particular data points are most important for the
>refinement of each parameter. Then send the diffractometer off to
>concentrate on the points where the derivatives are largest. That kind of
>requires that you can solve the problem before you collect the data, or at
>least much faster than it takes to collect the full dataset, so I doubt if
>it'll take off ;-) 
>
You are about to re-develop Optimal Experimental Design! This is a field of
mathematics since many decades. For a background in mathematics, read for
example:

Fedorov, V.V. "Theorija optimal'novo eksperimenta" , Moscow 1971
(in russian, "The Theory of optimal experiments"). Of course, it is
not such simple as you mean. You must optimize the error matrix, which follows
from some inversion of the data given by the measurements. And, in most
cases, the optimal designed experiment must have at least as many points as the
number of parameters used in the Rietveld refinement. Otherwise, the calulation
will diverge.

Optimal experimental design in XRPD was the topic of my doctoral thesis 1983
(see http://www.jbergmann.de/thesis.pdf). And, Optimal Experimental Design
was part of the BGMN Rietveld program from the beginning. I have just 
remanufactured it last weekend. I had not touched this code part for many
years, so it was somewhat out of function. See http://www.bgmn.de. We
hope to present new results of Optimal Experimental Design in near future.

Joerg Bergmann, Dresden
email@jbergmann.de

Date: Wed, 27 Feb 2002 17:22:31 +0100
To: rietveld_l@ill.fr
From: Armel Le Bail [alb@cristal.org]
Subject: Re: VCT 


>between a few possibilities of simple laws giving counting
>times as a function of diffracting angles, and that the Rietveld
>program would also have these laws inside.

One evident possible law could be one proportional to the inverse of
the Lorentz-polarisation factor, which is known in advance. In that
case, the Rietveld program would even be simplified and
calculation times would be slightly shorter... "D'une pierre deux
coups" (in French in the text). Remember that people refining
structures from single crystal data work on |F]squared data, generally.
We work on the same data, but they are modified by a number of
multiplicative factors that give undue weights to some parts of the powder
diffraction pattern. Removing the Lorentz-polarisation factor would
be all benefit.

No need to publish that in J. Appl. Cryst...
Take the idea with a copyleft (unless it was already published maybe ;-).

Armel

Date: Wed, 27 Feb 2002 18:09:44 +0100
To: rietveld_l@ill.fr
From: Jonathan WRIGHT [wright@esrf.fr]
Subject: Re: VCT 

>Armel wrote:
>But you may consider using the case where counts = counts ;-)

After you've finished the esd format refinement, you could plot
(y_obs/esd)**2 instead of normalised counts for y_obs and you have the same
thing (the intention is to plot something such that the esd is the square
root of what's on the y-axis, so you recover whatever the real counts were,
just in case I got the sums wrong or it wasn't immediately obvious).

>In that case, VCT or VS data would allow to give more weight
>than usually at increasing diffracting angle, in a way that mimics
>neutron data.

I thought an esd format already gives more weight? For seeing that clearly
you just need to modify your plot so that it represents the least squares
problem. Either use a weighted difference curve, or scale y_calc according
to y_obs above and then plot the straight difference. This will look
strange if you slow down for superstructure peaks! VCT is equivalent to
handling the changing incident intensity during a synchrotron experiment,
which is why I still think the Rietveld programmers cannot be blamed for
the method not catching on in the lab. Reading a scale factor with each
data point might be more convenient, but the gain is only aesthetic.

As Joerg has just pointed out, these ideas have been around for some time,
which was the reason I bought it up after the LaB6 data was posted. For
indexing and looking for superstructure peaks (and checking for a clean
pattern) it may not be appropriate, but otherwise it seems like a fairly
simple (and cheap) way to improve data quality?

Cheers,

Jon

PS: Some time ago I heard it said that anyone who is not using VCT data
with x-ray data for Rietveld refinement of structures should be shot. I
guess the person who said this is not doing much refereeing.

Date: Thu, 28 Feb 2002 11:57:29 +0100
To: rietveld_l@ill.fr
From: Alan Hewat [hewat@ill.fr]
Subject: Re: VCT

> >Which hardware vendor's software allows the collection of
> >variable count time data and the merging into a single
> >file with ESDs (or counts)?

I was hoping that some of the maintainers of the current Rietveld codes 
would answer this, but since they haven't I will comment :-)

Ever since multiple detectors have been used on neutron machines ie ever 
since the early 70's :-) we have had an additional weighting factor for 
each point that took into account the number of detectors used to obtain 
the average count for that point (since different points were measured a 
different number of times by different detectors). I suppose that similar 
things are done for the ESRF machine to which Kenneth Shankland refers, 
since it also uses multiple detectors. Each point is then recorded with two 
numbers - the average count and the number of times it was measured. Both 
GSAS and Fullprof accept this kind of data.

So it seems to me that it is commonly possible to refine data for which 
different regions of the pattern are collected with different counting 
times, and if that is all that is meant by "VCT" I am surprised to see so 
much discussion about it :-)

The wisdom of trying to measure some parts of the pattern more precisely 
because they are thought a priori to be more "important" is another matter 
:-) And to allow users to choose their own weights seems even more dangerous.

Alan.

Dr Alan W. Hewat, Diffraction Group Leader.
Institut Laue-Langevin, BP 156X Grenoble FRANCE 38042
fax (33)4.76.20.76.48 tel (33)4.76.20.72.13 (or .26 Mme Guillermet)
[hewat@ill.fr]  http://www.ill.fr/dif/AlanHewat.htm

From: "Sean X. Ouyang" [ouyang@mail.chem.tamu.edu]
To: [rietveld_l@ill.fr]
Subject: Re: VCT 
Date: Thu, 28 Feb 2002 10:06:23 -0600

The EVA software come with Bruker D8 machine allow you to merge multiple
data into a single one. You should configure your data collection with
overlapping for scaling. (ie. 5-40 10s, 35-65, 20s, 60-80 30s etc. and you
final collected data will have multiple sections). The overlap portions of
your data will be used for scaling and the data will be merged into a single
one. However, no esd provided.
BTW, how many Rietveld packages can handle data with esd? Could some body
point me to a data converting program which could import/export GSAS esd
data? (not powder2/3, which rounds off and flatten my data!).

Sean X. Ouyang Ph.D.
Department of Chemistry
Texas A&M University