An organization that is established as a means for achieving defined objectives has been referred to as a formal organization. Its design specifies how goals are subdivided and reflected in subdivisions of the organization. Divisions, departments, sections, positions, jobs, and tasks make up this work structure. Thus, the formal organization is expected to behave impersonally in regard to relationships with clients or with its members. According to Weber’s definition, entry and subsequent advancement is by merit or seniority. Each employee receives a salary and enjoys a degree of tenure that safeguards him from the arbitrary influence of superiors or of powerful clients. The higher his position in the hierarchy, the greater his presumed expertise in adjudicating problems that may arise in the course of the work carried out at lower levels of the organization. It is this bureaucratic structure that forms the basis for the appointment of heads or chiefs of administrative subdivisions in the organization and endows them with the authority attached to their position.
In contrast to the appointed head or chief of an administrative unit, a leader emerges within the context of the informal organization that underlies the formal structure. The informal organization expresses the personal objectives and goals of the individual membership. Their objectives and goals may or may not coincide with those of the formal organization. The informal organization represents an extension of the social structures that generally characterize human life – the spontaneous emergence of groups and organizations as ends in themselves.
In prehistoric times, man was preoccupied with his personal security, maintenance, protection, and survival. Now man spends a major portion of his waking hours working for organizations. His need to identify with a community that provides security, protection, maintenance, and a feeling of belonging continues unchanged from prehistoric times. This need is met by the informal organization and its emergent, or unofficial, leaders.
Leaders emerge from within the structure of the informal organization. Their personal qualities, the demands of the situation, or a combination of these and other factors attract followers who accept their leadership within one or several overlay structures. Instead of the authority of position held by an appointed head or chief, the emergent leader wields influence or power. Influence is the ability of a person to gain cooperation from others by means of persuasion or control over rewards. Power is a stronger form of influence because it reflects a person’s ability to enforce action through the control of a means of punishment.
Welcome to the e-learning lesson on Creating and Implementing a Data Collection Plan. Data collection is a crucial step in the process of measuring program outcomes. By measuring outcomes, an organization can better recognize the effectiveness and value of its programs, and pinpoint where changes or improvements need to be made. Before collecting data, your organization should have a solid understanding of the purpose of the program you wish to evaluate. You should have a working logic model that identifies your desired outcomes, the resources and activities necessary to accomplish these outcomes, and a detailed list of the specific measures you will take to implement them. Once this piece is complete, you can begin gathering relevant data through surveys, interviews, focus groups, or other methods.
Data collection happens before analysis and reporting.
Valid and reliable data is the backbone of program analysis. Collecting this data, however, is just one step in the greater process of measuring outcomes. The five steps include:
1. Identify outcomes and develop performance measures.
2. Create and implement a data collection plan (discussed in this lesson).
3. Analyze the data.
4. Communicate the results.
5. Reflect, learn, and do it again.
This lesson will illustrate effective options and techniques for data collection.
At the end of this lesson, you will be able to understand how to plan for and implement data collection for a specific program; identify the most appropriate and useful data collection methods for your purposes; and manage and ensure the integrity of the data you collect.
Data Collection Methods
Your data collection process will include attention to all the elements of your logic model: what resources you had available, what activities you actually provided, how many of each output you delivered, and to what degree you accomplished your outcomes. In collecting indicator data, you are likely to use one or more of these four methods: surveys, interviews or focus groups, observations, and record or document review. In selecting the best method for data collection, you will need to consider the type of information you need; the method’s validity and reliability; the resources you have available, such as staff, time, and money; and cultural appropriateness, or how well the method fits the language, norms, and values of the individuals and groups from whom you are collecting data.
Surveys are standardized written instruments that can be administered by mail, email, or in person.
The primary advantage of surveys is their cost in relation to the amount of data you can collect. Surveying generally is considered efficient because you can include large numbers of people at a relatively low cost. There are two key disadvantages: First, if the survey is conducted by mail, response rates can be very low, jeopardizing the validity of the data collected. There are mechanisms to increase response rates, but they will add to the cost of the survey. We will discuss tips for boosting response rates later in this lesson. Written surveys also don’t allow respondents to clarify a confusing question. Thorough survey pre-testing can reduce the likelihood that problems will arise.
Here are some examples of ways to use surveys:
• Track grassroots organizations’ use of and satisfaction with technical assistance services you provide.
• Survey all organizations receiving technical assistance to learn about changes in their fundraising tactics and the results of their efforts to raise more money.
Click here to download the “Technical Assistance Survey Template.” You can adapt this template for use in your program evaluation.
Interviews are more in-depth, but can be cost-prohibitive.
Interviews use standardized instruments but are conducted either in person or over the telephone. In fact, an interview may use the same instrument created for a written survey, although interviewing generally offers the chance to explore questions more deeply. You can ask more complex questions in an interview since you have the opportunity to clarify any confusion. You also can ask the respondents to elaborate on their answers, eliciting more in-depth information than a survey provides. The primary disadvantage of interviews is their cost. It takes considerably more time (and therefore costs more money) to conduct telephone and in-person interviews. Often, this means you collect information from fewer people. Interview reliability also can be problematic if interviewers are not well-trained. They may ask questions in different ways or otherwise bias the responses.
Here are some examples of ways to use interviews:
• Talk to different grassroots organizations to learn about the way in which they are applying new knowledge of partnership development.
• Interview individuals within an organization to explore their perceptions of changes in capacity and ability to deliver services.
Focus groups are small-group discussions based on a defined area of interest.
While interviews with individuals are meant to solicit data without any influence or bias from the interviewer or other individuals, focus groups are designed to allow participants to discuss the questions and share their opinions. This means people can influence one another in the process, stimulating memory or debate on an issue. The advantage of focus groups lies in the richness of the information generated. The disadvantage is that you can rarely generalize or apply the findings to your entire population of participants or clients. Focus groups often are used prior to creating a survey to test concepts and wording of questions. Following a written survey, they are used to explore specific questions or issues more thoroughly.
Here are some examples of ways to use focus groups:
• Hold a structured meeting with staff in a community-based organization to learn more about their grants management practices, what worked during the year, and what did not.
• Conduct a discussion with staff from several organizations to explore their use of computer technology for tracking financial data.
Observations can capture behaviors, interactions, events, or physical site conditions.
Observations require well-trained observers who follow detailed guidelines about whom or what to observe, when and for how long, and by what method of recording. The primary advantage of observation is its validity. When done well, observation is considered a strong data collection method because it generates firsthand, unbiased information by individuals who have been trained on what to look for and how to record it. Observation does require time (for development of the observation tool, training of the observers, and data collection), making it one of the costlier methods.
Here are some examples of ways to use observations:
• Observe individuals participating in training to track the development of their skill in the topic.