27 March 2005

Regional trends in primary school enrollment

In the previous two weeks, country-level data on primary school enrollment and on the gender disparity in primary school was presented on this site. This week we look at regional trends in primary school enrollment from 1980 to 2000. For this purpose I adopt the regional groupings used by UNICEF, which assigns countries to eight regions according to shared cultural and economic characteristics.
  • East Asia and the Pacific
  • South Asia
  • Eastern Europe, Commonwealth of Independent States (CIS)
  • Middle East and North Africa
  • Eastern and Southern Africa
  • West and Central Africa
  • Latin America and the Caribbean
  • Industrialized countries
UNICEF regions
Map of the world showing UNICEF regions
Source: UNICEF website, March 2005

The data for the trend analysis is from two sources: (a) UNESCO's Global Education Digest 2004, with data for the years 1998 to 2001; and (b) statistical tables from the website of the UNESCO Institute for Statistics that cover the years 1970 to 1997.

At the global level, the primary school net enrollment ratio (NER) has increased from 78% in 1980 to 88% in 2000. Over the same period, the disparity between boys' and girls' enrollment has decreased from 12% to 4%.

Primary school net enrollment ratio, 1980-2000
Graph showing primary school net enrollment rate between 1980 and 2000 for each UNICEF region
Data sources: UNESCO Institute for Statistics (UIS). 2004. Global Education Digest 2004. Montreal: UIS. - World Education Indicators, UNESCO Institute for Statistics, March 2005.

A look at regional trends reveals striking differences. Sub-Saharan Africa has the lowest enrollment rates, with school participation at the same level today as it was in the 1980s. In 2000, the primary school net enrollment ratio for boys and girls combined was 55% in West and Central Africa and 64% in Eastern and Southern Africa. In contrast, the industrialized countries, Latin America, and East Asia are close to achieving the goal of universal primary education for all children.

Girls continue to be disadvantaged in South Asia and in West and Central Africa. Countries in the Middle East and North Africa have made remarkable progress towards gender parity since the 1980s.

Friedrich Huebler, March 2005, Creative Commons License

20 March 2005

Gender disparity in primary school

All UN member states have agreed to "eliminate gender disparity in primary and secondary education, preferably by 2005, and in all levels of education no later than 2015," as stated in the Millennium Development Goals. However, statistics from UNESCO's Global Education Digest 2004 make clear that gender parity in primary education cannot be achieved in 2005. The map below shows the gap between the primary school net enrollment ratios (NER) of boys and girls in 2001/02, the latest year for which data is available.

Gender disparity in primary school: male-female net enrollment ratio, 2001/2002
Map of the world showing male-female disparity in primary school net enrollment rate for each country
Data source: UNESCO Institute for Statistics (UIS). 2004. Global Education Digest 2004. Montreal: UIS.

Girls continue to be disadvantaged in many countries of Africa, the Middle East, South Asia, and East Asia. In three countries, the NER of girls was more than 20 percentage points below that of boys: Benin (26% difference in NER), Yemen (24%), and Chad (23%). In 14 more countries, the gender disparity in primary school enrollment was between 10 and 20 percentage points: Pakistan (20%), Cote d'Ivoire (19%), Liberia (18%), Togo (16%); Guinea, Guinea-Bissau, and India (15%); Iraq and Equatorial Guinea (14%); Niger (13%); Mali and Burkina Faso (12%); Burundi and Ethiopia (11%). In 20 countries, the gender disparity was between 5 and 10 percentage points, and in 33 countries it was between 1 and 5 percentage points.

While gender disparity in primary school is mostly to the disadvantage of girls, there are also countries where boys are less likely to be enrolled. In four countries, the NER of girls was 5 ore more percentage points above that of boys: Saint Kitts and Nevis (9% difference in NER), Lesotho (6%); Namibia and Netherlands Antilles (5%). Another 19 countries had female-male disparities between 1 and 5 percentage points.

Related articles
Friedrich Huebler, 20 March 2005 (edited 12 October 2008), Creative Commons License
Permanent URL: http://huebler.blogspot.com/2005/03/gender-disparity-in-primary-school.html

15 March 2005

Primary school enrollment

At the United Nations Millennium Summit in 2000, all UN member states agreed to reach a set of Millennium Development Goals (MDG) by the year 2015. The MDGs address poverty in its many dimensions and one goal is to "ensure that, by 2015, children everywhere, boys and girls alike, will be able to complete a full course of primary schooling."

The map below was created with data from the Global Education Digest 2004, published by the UNESCO Institute for Statistics. It shows the primary school net enrollment ratios in 174 countries for which data was available. The net enrollment ratio (NER) is the share of children of primary school age that are currently enrolled in school. When the NER is at 100%, all children are attending primary school at the proper age.

Primary school net enrollment ratio, 2001/2002
Map of the world showing primary school net enrollment rate for each country
Data source: UNESCO Institute for Statistics (UIS). 2004. Global Education Digest 2004. Montreal: UIS.

The most recent data in the Global Education Digest covers the school year 2001/2002. In countries without data for 2001/02, I used enrollment ratios from previous years, as far back as 1998/99. The map shows that many countries in the world are close to reaching the goal of primary education for all children. Many other countries, primarily in Africa, have to make significant progress during the next ten years to reach this goal. In 17 countries - all in Africa and the Middle East - not more than six out of ten children of primary school age were enrolled in primary school: Angola (30%), Djibouti (34%), Niger (34%), Democratic Republic of the Congo (35%), Burkina Faso (35%), Mali (38%), Eritrea (43%), Guinea-Bissau (45%), Sudan (46%), Ethiopia (46%), Burundi (53%), Tanzania (54%), Comoros (55%), Senegal (58%), Chad (58%), Saudi Arabia (59%), and Mozambique (60%).

Friedrich Huebler, 15 March 2005, Creative Commons License

10 March 2005

Guide to integrating Stata and external text editors (archived version)

As of 27 April 2008, this document is no longer maintained and has been replaced by a new guide to integrating Stata and external text editors. The programs described below were replaced by new, precompiled versions that no longer require the installation of AutoIt. Please change your links to the new guide: http://huebler.blogspot.com/2008/04/stata.html.



Table of contents

Updates to this guide
Introduction
Requirements
Script 1: Run a do-file from an external editor
Script 2: Run individual commands from an external editor
Supported editors
History of the scripts
Known problems
Acknowledgments

Updates to this guide

Introduction

Stata is a statistical package for research professionals of all disciplines, from biostatisticians to political scientists. Stata is available for the Windows, Macintosh, and Unix platforms. For more information, go to the official Stata website.

This guide addresses some shortcomings of the text editor that is part of the Stata package and that is used to write programs, or do-files, as they are called in Stata. The Do-file Editor of Stata has one advantage over external editors: it is fully integrated with Stata and commands can be executed directly from the editor, with a keyboard shortcut or the click of a button. External editors often have a richer set of features, including syntax highlighting, but they lack integration with Stata.

This document shows how external text editors can be integrated with Stata in a way similar to the Do-file Editor. This can be accomplished with AutoIt, a free, open source scripting language designed for automating tasks in Windows. (AutoIt is not available for other platforms.)

Requirements

Software needed:
  • Stata version 7, 8, 9, or 10. Older versions of Stata may also work but have not been tested.
  • AutoIt: Download and install on your computer. At the time of writing, the latest version was v3.1.1, released in April 2005.
  • Text editor: The scripts were tested with EmEditor (the latest version at the time of writing was 4.13) and TextPad but they should work with other editors after minor modifications. A list of supported editors can be found near the end of this guide.
Two AutoIt scripts are shown below. The first script runs an entire do-file from a text editor. The second script runs selected lines from an editor. The following steps describe how the scripts can be used to integrate an external text editor with Stata.
  1. Modify the scripts if necessary (see the comments below) and save them as text files with the extension AU3, for example as rundo.au3 and rundolines.au3.
  2. Compile the scripts: There are two ways to do this (see the AutoIt help file), either with the "Compile script to .exe" program in the AutoIt v3 program group, or by right-clicking on an .au3 file and selecting "Compile script" from the context menu.
  3. Find a way to call the compiled scripts (in this example, rundo.exe and rundolines.exe) from your text editor with a keyboard shortcut, a menu entry, or an icon in the toolbar.

In the case of EmEditor, the scripts can be called with the External Tools feature. Go to the Tools menu and select External Tools and then Customize Tools. Click on New and enter the information for the first script: Title (e.g., "Run do-file"), Command (the full path to the compiled script), and Icon Path (e.g., the path to the Stata executable, but any other icon can be used). [Additional step for Script 1 (run do-file): Enter the text "$(Path)" (with quotation marks) in the field Arguments. This sends the path of the saved do-file to the AutoIt script.] Click on OK and again OK. Go to the Tools menu and select Customize Toolbars. Select the desired toolbar and click on Customize. Add the new icon to the toolbar and click on Close. Repeat the steps for the second script; select a different icon so that the two scripts can be distinguished easily. To call a script click on one of the two new icons in the EmEditor toolbar.

With TextPad the scripts can also be called by adding them to the toolbar. Go to Configure, Preferences, Tools, click Add, and browse to the compiled AutoIt script. [The parameter $File is needed for Script 1 only.] Then right-click on the toolbar, select Customize, Commands, add an icon for the script, and move the icon to the TextPad toolbar.


Script 1: Run a do-file from an external editor (download this script)

; AutoIt v3 script to run a Stata do-file from an external text editor.
; Version 2.2, Friedrich Huebler (fhuebler at gmail.com), 26 September 2005.
; Updated by Nicholas Winter (nw53 at cornell.edu), 25 May 2005.
; Adapted from a script by Dimitriy V. Masterov
; (dvmaster at lily.src.uchicago.edu), 23 June 2004.
; AutoIt is available at http://www.autoitscript.com/autoit3/.

; Declare variables
Global $statapath, $statawin, $dofile

; NOTE: Edit $statapath and $statawin before script is compiled
; Path to Stata executable
$statapath = "C:/Program Files/Stata9/wsestata.exe"
; Title of Stata window
$statawin = "Stata/SE 9.1"

; EXAMPLE: For Intercooled Stata 8.2 delete preceding block and use commands below
; $statapath = "C:/Program Files/Stata8/wstata.exe"
; $statawin = "Intercooled Stata 8.2"

; NOTE: Edit this block of commands to match the editor used
; EmEditor or TextPad: path of do-file is passed to AutoIt
$dofile = $CmdLine[1]

; Alternative method to obtain path of do-file with EmEditor
; Get path of do-file from title of EmEditor window
; $dofile = WinGetTitle("")
; Remove unwanted text from window title to keep path only
; $dofile = StringReplace($dofile," - EmEditor","")

; Alternative method to obtain path of do-file with TextPad
; Get path of do-file from title of TextPad window
; $dofile = WinGetTitle("")
; Remove unwanted text from window title to keep path only
; $dofile = StringReplace($dofile,"TextPad - ","")
; $dofile = StringReplace($dofile,"[","")
; $dofile = StringReplace($dofile,"]","")

; If more than one Stata window is open, the window
; that was most recently active will be matched
Opt("WinTitleMatchMode",2)

; Reduce SendKeyDelay and WinWaitDelay to speed up script
Opt("SendKeyDelay", 1)
Opt("WinWaitDelay", 200)

; Check if Stata is already open, run it if not
If WinExists($statawin) Then
  WinActivate($statawin)
  WinWaitActive($statawin)
  ; Activate Stata Command Window and select text (if any)
  Send("^4")
  Send("^a")
  ; Run saved do-file
  ; Double quotes around $dofile needed in case path contains blanks
  ClipPut("do " & '"' & $dofile & '"')
  ; Pause avoids problem with clipboard, may be AutoIt or Windows bug
  Sleep(100)
  Send("^v" & "{Enter}")
Else
  Run($statapath)
  WinWaitActive($statawin)
  ; Activate Stata Command Window
  Send("^4")
  ; Run saved do-file
  ; Double quotes around $dofile needed in case path contains blanks
  ClipPut("do " & '"' & $dofile & '"')
  ; Pause avoids problem with clipboard, may be AutoIt or Windows bug
  Sleep(100)
  Send("^v" & "{Enter}")
EndIf

Comments on script 1: In most cases, the script must be edited in two places before it can work on other computers:

  1. The variables $statapath and $statawin must match your configuration. $statapath contains the location of the Stata executable, which can be found with the Windows Explorer. The text in the variable $statawin is taken from the title of the main Stata window.
  2. The variable $dofile stores the location of the do-file. With EmEditor and TextPad, the path can be passed to the AutoIt script via the command line; this may also be possible with other editors. An alternative is to extract the path from the title of the editor window; the script shows how this can be done with EmEditor and TextPad. The AutoIt Window Info tool, a component of the AutoIt package, can help you find some of the information that is required with the alternative approach (window title, etc.).

What the script does: If Stata is already open, the script sends the path of the do-file from the editor to the Command window in Stata, preceded by "do" and followed by Enter. If more than one instance of Stata is open, the do-file is executed in the Stata window that was most recently active. If Stata is not open, the script starts Stata and then runs the do-file.

Important limitation: A do-file must be saved before running script 1 because the script looks for the saved version of the document that is currently being edited. An alternative would be to select the entire text and run script 2 below, which passes selected lines to Stata.


Script 2: Run individual commands from an external editor (download this script)

; AutoIt v3 script to run Stata commands from an external text editor.
; Version 2.2, Friedrich Huebler (fhuebler at gmail.com), 26 September 2005.
; Updated by Nicholas Winter (nw53 at cornell.edu), 25 May 2005.
; Adapted from a script by Eva Poen (eva.poen at unisg.ch), 27 June 2004.
; AutoIt is available at http://www.autoitscript.com/autoit3/.

; Declare variables
Global $statapath, $statawin, $commands, $tempfile, $tempfile2

; NOTE: Edit $statapath and $statawin before script is compiled
; Path to Stata executable
$statapath = "C:/Program Files/Stata9/wsestata.exe"
; Title of Stata window
$statawin = "Stata/SE 9.1"

; EXAMPLE: For Intercooled Stata 8.2 delete preceding block and use commands below
; $statapath = "C:/Program Files/Stata8/wstata.exe"
; $statawin = "Intercooled Stata 8.2"

; If more than one Stata window is open, the window
; that was most recently active will be matched
Opt("WinTitleMatchMode",2)

; Reduce SendKeyDelay and WinWaitDelay to speed up script
Opt("SendKeyDelay", 1)
Opt("WinWaitDelay", 200)

; Clear clipboard
ClipPut("")
; Copy selected lines from editor to clipboard
Send("^c")
; Pause avoids problem with clipboard, may be AutoIt or Windows bug
Sleep(100)
$commands = ClipGet()

; Terminate script if nothing selected
If $commands = "" Then
  Exit
EndIf

; Create file name in system temporary directory
$tempfile = EnvGet("TEMP") & "\statacmd.tmp"

; Open file for writing and check that it worked
$tempfile2 = FileOpen($tempfile,2)
If $tempfile2 = -1 Then
  MsgBox(0,"Error: Cannot open temporary file","at [" & $tempfile & "]")
  Exit
EndIf

; Write commands to temporary file, add CR-LF at end
; to ensure last line is executed by Stata
FileWrite($tempfile2,$commands & @CRLF)
FileClose($tempfile2)

; Check if Stata is already open, run it if not
If WinExists($statawin) Then
  WinActivate($statawin)
  WinWaitActive($statawin)
  ; Activate Stata Command Window and select text (if any)
  Send("^4")
  Send("^a")
  ; Run temporary file
  ; Double quotes around $tempfile needed in case path contains blanks
  ClipPut("do " & '"' & $tempfile & '"')
  ; Pause avoids problem with clipboard, may be AutoIt or Windows bug
  Sleep(100)
  Send("^v" & "{Enter}")
Else
  Run($statapath)
  WinWaitActive($statawin)
  ; Activate Stata Command Window
  Send("^4")
  ; Run temporary file
  ; Double quotes around $dofile needed in case path contains blanks
  ClipPut("do " & '"' & $tempfile & '"')
  ; Pause avoids problem with clipboard, may be AutoIt or Windows bug
  Sleep(100)
  Send("^v" & "{Enter}")
EndIf

Comments on script 2: In most cases, the script must be edited before it can work on other computers: The variables $statapath and $statawin must match your configuration. $statapath contains the location of the Stata executable, which can be found with the Windows Explorer. The text in the variable $statawin is the title of the main Stata window.

What the script does: The lines that are selected in the editor are saved in a temporary file. If Stata is already open, the script sends the path of the temporary file to the Command window in Stata, preceded by "do" and followed by Enter. If more than one instance of Stata is open, the commands are executed in the Stata window that was most recently active. If Stata is not open, the script starts Stata and then runs the selected commands.


Supported editors

The following editors are confirmed to work with the AutoIt scripts above:
  • Crimson Editor: Salmai Qari and Matthew Forbes informed me that the scripts can be linked to Crimson Editor as follows:
    1. Select "Conf. User Tools" from the "Tools" menu in Crimson Editor.
    2.a. Rundo script: Command: path to rundo.exe, e.g. C:\ado\rundo.exe, Argument: $(FilePath).
    2.b. Rundolines script: Command: path to rundolines.exe, e.g. C:\ado\rundolines.exe.
    3. Define Hot Keys as desired.
  • EditPlus: Thanks to Raul Sanchez for testing the scripts.
  • EmEditor: If you use EmEditor you may be interested in a Stata syntax file that I created for this editor. The syntax file is available free of charge at the EmEditor User Files page.
  • TextPad
  • UltraEdit: Robert White sent me these instructions for the configuration of UltraEdit. Go to Advanced and then Tool Configuration. For the rundo script, the command line should read:
    c:\stata\rundo.exe "%f"
    For the rundolines script, the command line should read:
    c:\stata\rundolines.exe do "%f"
    Next, set the working directory (for instance, C:\DATA) and choose a name for the menu item (for instance, rundo or rundolines). Make sure to check "Windows Program" and uncheck "Save Active File". To finish, click on the Insert button at right. rundo (rundolines) now appears at the bottom of the Advanced menu. Clicking on these entries calls Stata and runs the AutoIt executables.
    Marcos Delprato informed me that the rundo and rundolines commands can also be added to the UltraEdit toolbar. He also pointed out that the "Capture Output" option in the Output window has to be unchecked when the scripts are set up with UltraEdit.
Other authors have published modified versions of the scripts for other editors.
  • Any editor, including gVim: Jeffrey Arnold describes his modified version of the AutoIt scripts in a post to Statalist. In addition to other changes, he uses an INI file in text format that makes it unnecessary to specify the location of the Stata executable and the the title of the Stata window in the AutoIt script itself.
  • Hidemaru: Masakazu Hojo wrote a program called Hidomaru to integrate Stata with the Japanese Hidemaru editor.
  • jEdit: Dimitriy V. Masterov describes in a post to Statalist how to integrate Stata and jEdit.
  • Notepad++: Keith Kranker wrote a guide that describes how to run Stata files from Notepad++.
  • Notepad++: The original scripts are inconvenient if the Stata do-files are written with a semicolon as delimiter. (I personally use a carriage return as delimiter.) Ansgar Wolsing offers modified scripts for users who prefer a semicolon as delimiter.
  • Notepad++ and other editors: Stefan Gawrich modified the scripts so that do-files can be run from the start of the document to the current cursor position.
  • SciTE: Ari Friedman sent a message to Statalist in which he explains how to integrate Stata and SciTE.
  • Vim: Dimitriy V. Masterov describes in a post to Statalist how to integrate Stata and Vim.
  • WinEdt: Changhwan Kim and Haebong Woo explain how to adapt the AutoIt scripts to integrate WinEdt with Stata.
I would like to thank everyone who tested the scripts with other editors. Please write to me at fhuebler@gmail.com if you use the scripts with additional editors so that I can update this list. Some editors that can be used with Stata are described at "Some notes on text editors for Stata users".

History of the scripts

  • July 2004: First version of rundolines.au3 sent to the Stata mailing list (see Statalist archive).
  • October 2004: First version of this guide, with both scripts, released as part of "Some notes on text editors for Stata users".
  • March 2005: Scripts revised to work with Stata 7 and 8.
  • March 2005: Expanded guide added to my blog on education statistics.
  • April 2005: Guide and scripts added to my Web site, with direct download links.
  • 6 June 2005: Version 2.0 of scripts released, with support for Stata 9. The scripts are now compatible with Stata 7, 8, and 9. Scripts work with more than one instance of Stata. Scripts no longer rely on the built-in editor of Stata. New version of this guide.
  • 8 June 2005: Version 2.1 of scripts released. Fixed problem with the clipboard that was reported by some users: scripts no longer attempt to restore clipboard or text in Stata Command window.
  • 30 June 2005: Version 2.1.1 of rundolines script released, with minor bug fix related to clipboard.
  • 26 September 2005: Version 2.2 of scripts released. The scripts are now compatible with Stata 9.1, released on 15 September 2005.
  • 22 June 2006: New section added to this guide: Supported editors.
  • 28 January 2007: The list of supported editors now includes EmEditor, gVim, Hidemaru, jEdit, Notepad++, SciTE, TextPad, Vim, UltraEdit, and WinEdt.
  • 26 April 2008: The following editors have been confirmed to work with the scripts: Crimson Editor, EditPlus, EmEditor, gVim, Hidemaru, jEdit, Notepad++, SciTE, TextPad, Vim, UltraEdit, and WinEdt.
  • 27 April 2008: Version 3 of the rundo and rundolines programs released. This document is no longer maintained and has been replaced by a new guide to integrating Stata and external text editors.

Known problems

In rare cases, no command is passed to Stata; if this happens the script should be called a second time, after which it will usually work. This problem is due to the way the clipboard is handled and I don't know if this is a bug in AutoIt or in Windows. If a script fails, you can increase the time in the Sleep commands and recompile the scripts. For example, use Sleep(200) instead of Sleep(100), where the number indicates the pause in milliseconds. Up to version 2.0 the scripts attempted to restore the contents of the clipboard and/or the Stata Command window. I removed this feature starting with version 2.1 because of the clipboard problem. Version 2.0 of the scripts can still be downloaded but I strongly recommend the most recent version instead: rundo20.zip, rundolines20.zip.

Acknowledgments
  • I am building on previous work by Dimitriy Masterov and Eva Poen. Masterov demonstrated in a post to the Stata mailing list how a do-file can be executed from Vi (see Statalist archive). Poen followed up by showing how individual lines can be executed from WinEdt (see Statalist archive).
  • Nicholas Cox prompted me to write this guide as a contribution to "Some notes on text editors for Stata users," a FAQ that he maintains.
  • Amanda Tzy-Chyi Yu brought to my attention that one of the scripts did not work with Stata 7 and helped by testing new versions of the scripts.
  • Nicholas Winter updated the scripts so that they work with Stata 9. He also made the scripts more efficient by finding a solution that does not require the Do-file Editor of Stata and suggested using Sleep() to avoid the problem with the clipboard.
Please leave comments and corrections at the "Post a comment" link below or send a message to fhuebler@gmail.com. Thank you.

Related articlesFriedrich Huebler, 10 March 2005 (edited 1 July 2012), Creative Commons License
Permanent URL: http://huebler.blogspot.com/2005/03/integrating-stata-and-external-text.html

08 March 2005

Recommended freeware

Friedrich Huebler, 8 March 2005 (edited 1 February 2009), Creative Commons License
Permanent URL: http://huebler.blogspot.com/2005/03/recommended-freeware.html