- Nazwa przedmiotu:
- Fundamentals of information theory
- Koordynator przedmiotu:
- Prof. dr hab. inż. Artur Przelaskowski
- Status przedmiotu:
- Obowiązkowy
- Poziom kształcenia:
- Studia I stopnia
- Program:
- Computer Science and Information Systems
- Grupa przedmiotów:
- Obligatory
- Kod przedmiotu:
- .
- Semestr nominalny:
- 2 / rok ak. 2024/2025
- Liczba punktów ECTS:
- 4
- Liczba godzin pracy studenta związanych z osiągnięciem efektów uczenia się:
- 1. contact hours - 50 h; including
a) attendance at lectures - 30 h
b) attendance at laboratories - 15 h
c) consultations - 5 h
2. students' own work - 60 h; including
a) reading the literature - 10 h
b) preparation for tests - 20 h
c) preparation for laboratory classes - 30 h
Total 110 h, which corresponds to 4 ECTS points.
- Liczba punktów ECTS na zajęciach wymagających bezpośredniego udziału nauczycieli akademickich:
- 1. lecture attendance - 30 h
2. attendance at laboratories - 15 h
3. consultations - 5 h
Total 50 h, which corresponds to 2 pts. ECTS
- Język prowadzenia zajęć:
- angielski
- Liczba punktów ECTS, którą student uzyskuje w ramach zajęć o charakterze praktycznym:
- -
- Formy zajęć i ich wymiar w semestrze:
-
- Wykład30h
- Ćwiczenia0h
- Laboratorium15h
- Projekt0h
- Lekcje komputerowe675h
- Wymagania wstępne:
- -
- Limit liczby studentów:
- .
- Cel przedmiotu:
- Course objective: to provide knowledge and skills regarding the fundamentals of information theory, including effective representations of signals and how to model the sources, counting and understanding of information, used to form effective syntactic, semantic and pragmatic communication
- Treści kształcenia:
- Lecture: The concept of a physical signal, as the effect of a measurement in a specific layout-system, and its analogue representation: definition of the measurement process, definition of physical, organisational conditions, problem of intentionality and randomness, receiving and transmitting/transmitting.
Examples of signals, their mathematical representation, spaces, bases, differentiation of signal characteristics, quality measures.
Review of simple methods of analysis of analogue signals, their amplitude, frequency, phase, time-frequency, scalable representations; basic shaping filters.
Digital signals, analogue-to-digital converters, sampling and quantisation rules, uniform and adaptive schemes, reconstruction rules for analogue signals, quality control - examples of aliasing distortion, Gibbs effects etc. of audio signals, images.
Understanding the concept of information: intuition and colloquial definitions, examples, discussion of the crucial importance of information in the context of developments in modern technology, physics, biology, cosmology, etc. (Wiener: information is neither matter nor energy; Bateson: information is the difference that makes the difference); the concept of quantum information (the state of a quantum system).
Fundamental definition of the concept of information, characteristics of information and ways to describe them; model of information transmission, channel, sender-receiver scheme; problem of objectification of subjective user models, optimisation criteria.
C.E. Shannon's mathematical (statistical) theory of information: source models, measures of information quantity, basic coding theorems; system and systemic implications; other syntactic theories of information: combinatorial and algorithmic; Gibbs entropy (a measure of disorder in a closed system of particles in equilibrium with respect to the probability distribution of energy).
Unambiguously decodable codes: bijection conditions, examples of codes, optimal codes.
Analytical theory of Kolmogorov information networks, notion of epsilon-entropy, examples of its determination, practical importance of this theory in the construction of efficient compression methods with information selection.
Semantic and pragmatic theories of information, emphasising the importance and cognitive qualities of the elements of information transmission; examples: the first linguistic concepts of Carnap and Bar-Hillel (the greater the number of sentences a word can imply in a language model, the word contains more semantic information); Floridi and the cognitive, philosophical concept of information (meaning, truth and knowledge); the comprehensive model of information theory (Stanford); the truth problem in information theory; models of semantic information generation; representations and measurements of semantic information.
Information system modelling: observed objects (measured, described by ontology, cognised) in a specific environment (specific conditions) - measurement of object properties forming information - cognition by perceiving information - decision-making with reference to domain knowledge - execution of intentions by acting intelligently on observed (or analogous) objects; use of information system to build knowledge (induction) for intelligent realisation of specific goals (deductive method).
Realistic examples of information theory applications: multimedia applications (browsing resources by content, interactive transmissions, object recognition, interpretation of their state, behavioural dynamics, development trends), medical informatics systems (clinical decision support, selection of therapy form, interpretation of diagnosis), reconstruction of objects on the basis of rare measurements/representations (problem of targeted measurements, random measurements with knowledge model and representative projections).
Laboratory (subject areas covered in exercise and project form):
1. Signal measurement and reconstruction: observation - viewing the world, searching for sources of information in the context of a specific application model; signal acquisition - sensors, acquisition rules, measurement noise and distortion; signal shaping (preprocessing, filtering), quality control; information reconstruction (inverse problem, limited number of measurements, information criteria)
2. Information representation: conceptual and experimental reduction of data redundancy; defining and understanding information, modelling, counting, coding and expressing/expressing information
3. Information extraction: information reconstruction, processing for extraction, content approximation, compression with information selection (image, sound)
4. Information retrieval: remote browsing of vast resources to find specific information for a specific application, data indexing - attributes, features, object lists, mechanisms for efficient retrieval of specific content/images/sounds/multimedia, semantic descriptors, semantic similarity measures, search selectivity - precision, recall, success rate, browsers, search tests
5. Use of information: characteristics and experimental verification of the usefulness of information transfer; reference to practical applications and real benefits of information use; reliable evaluation of usefulness, search for possibly significant, meaningful benefits of use.
Form of implementation:
- exercise/project (tools/software, experiments, report according to the basic model) carried out in groups (max. 7 persons with possibility of individual work);
- possible extensions in terms of software/tool and scope of analysis/experiments, theory of the studied issue (new developments or applications, reference to larger-scale experiments, creative conclusions);
- scheme: in the introductory class (1 class in the cycle) presentation of the problem, explanation of the basic pattern of the implementation of the exercise, discussion of problems, then consultations in teams (2-3 labs, depending on the needs, stationary or remote form) and completion of the next exercise (through discussion of the prepared report on the implementation of the exercise - list of activities, theoretical background of the issue, description of tools and performed experiments, results and conclusions);
- evaluation of individual labs on a scale of 0-8 points (5*8=40 points); the threshold for passing a lab is 21 points.
- Metody oceny:
- A student may receive:
* up to 10 points for activity:
- discussion of problems undertaken,
- solving additional tasks (from the textbook, indicated lectures and exercises)
*40 pts for laboratory exercises (5x8 pts)
*50 points for the final colloquium.
The overall pass mark for the course is 51 pts, and the distribution of successive pass marks is a sequence of 61, 71, 81 and 91 pts, including the additional requirement to pass the laboratory for min. 21pts
- Egzamin:
- nie
- Literatura:
- 1. Gareth A. Jones and J. Mary Jones, Information and Coding The- ory, Springer, 2000.
2. T.M. Cover, J.A. Thomas, "Elements of Information Theory", 2nd Edition, Wiley & Sons, Hoboken, New Jersey, 1991
3. David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
4. I.H. Witten, A. Moffat , T.C. Bell, „Managing Gigabytes. Compressing and Indexing Documents and Images ”, Morgan Kaufmann Publishers, 2nd edition, 1999
5. K. Sayood, „Introduction to Data Compression”, Third Edition, Morgan Kaufmann Publishers, 2006 .
6. M. Nelson, „The Data Compression Book”, 2nd edition, MIS:Press,1995
7. A. Neubauer, J. Freudenberger, V.Kuhn, „Coding theory. Algorithms, architectures and Applications ”, Wiley, 2007
8. D.J.C. MacKay: „Information Theory, Inference, and Learning Algorithms”, Cambridge University Press, Cambridge, UK, 2003
9. A.Abbate, C.M.DeCusatis, P.K.Das, Wavelets and subbands. Fundamentals and applications, Birkhauser 2002
10. P.J.Van Fleet, Discrete wavelet transformations. An elementary approach with applications, Wiley 2008
11. J.Kovacevic, V.K.Goyal, M.Vetterli, Fourier and wavelet signal processing, 2013
12. M.Vetterli, J. Kovacevic,V.K.Goyal, Foundations of signal processing, 2012
13. Signal processing: Fourier and wavelet representations, 2012
14. M.W.Frazier, An introduction to wavelets through linear algebra, Springer-Verlag, 1999
15. Ming Li, Paul Vitanyi, An Introduction to Kolmogorov Complexity and Its Applications. Springer, 1997.
- Witryna www przedmiotu:
- .
- Uwagi:
- .
Efekty uczenia się
Profil ogólnoakademicki - wiedza
- Charakterystyka W01
- .
Weryfikacja: .
Powiązane charakterystyki kierunkowe:
Powiązane charakterystyki obszarowe:
Profil ogólnoakademicki - umiejętności
- Charakterystyka U01
- .
Weryfikacja: .
Powiązane charakterystyki kierunkowe:
Powiązane charakterystyki obszarowe:
- Charakterystyka U02
- .
Weryfikacja: .
Powiązane charakterystyki kierunkowe:
Powiązane charakterystyki obszarowe:
- Charakterystyka U03
- .
Weryfikacja: .
Powiązane charakterystyki kierunkowe:
Powiązane charakterystyki obszarowe:
- Charakterystyka U04
- .
Weryfikacja: .
Powiązane charakterystyki kierunkowe:
Powiązane charakterystyki obszarowe:
Profil ogólnoakademicki - kompetencje społeczne
- Charakterystyka K01
- .
Weryfikacja: .
Powiązane charakterystyki kierunkowe:
Powiązane charakterystyki obszarowe: