Python 3.6 - Dictionary Ordering

Fred Baptiste
A free video tutorial from Fred Baptiste
Professional Developer and Mathematician
4.8 instructor rating • 4 courses • 37,561 students

Learn more from the full course

Python 3: Deep Dive (Part 1 - Functional)

Variables, Functions and Functional Programming, Closures, Decorators, Modules and Packages

44:37:26 of on-demand video • Updated August 2020

  • An in-depth look at variables, memory, namespaces and scopes
  • A deep dive into Python's memory management and optimizations
  • In-depth understanding and advanced usage of Python's numerical data types (Booleans, Integers, Floats, Decimals, Fractions, Complex Numbers)
  • Advanced Boolean expressions and operators
  • Advanced usage of callables including functions, lambdas and closures
  • Functional programming techniques such as map, reduce, filter, and partials
  • Create advanced decorators, including parametrized decorators, class decorators, and decorator classes
  • Advanced decorator applications such as memoization and single dispatch generic functions
  • Use and understand Python's complex Module and Package system
  • Idiomatic Python and best practices
  • Understand Python's compile-time and run-time and how this affects your code
  • Avoid common pitfalls
English [Auto] Hi, I want to talk a little bit about dictionary ordering and Python three point six, a new implementation of DECT came out in three point six. And one of the features of that about from being supposedly faster and more compact is that key ordering is retained. And what does that mean? That just means that the order in which the keys are brought back when we iterate through the keys or the items or the values, is in the same order in which the keys were inserted, the elements were inserted into the dictionary. So basically this replaces the old dict, which is in the collections module, in the standard library. And I'll come back to that as well and take a look at all the dict and see if we can actually do everything we need using just plain dictionaries. But a few caveats first. This is in three point six only, so it's going to have to be three point six and higher. Secondly. In three point six, it's not an official fact, in other words, it wasn't guaranteed that Keyes would remain, that dictionaries would be ordered it. It doesn't guarantee that in three point seven. However, it does it does guarantee that if you iterate a dictionary, you will get it back in the same order in which you added the keys to the dictionary and it's supposed deletions as well. So if your code is going to be three point six or higher, then you can certainly leverage that feature. But if your code has to run on both three point six and higher and lower than three point six, and you need to rely on ordering in dictionaries, then you still need to use the order dict. Otherwise your code will break invasion's prior to three point six. So just be careful with that. But as long as you're writing for three point six and higher, this is safe. Even though it's not official for three point six, it will become official for three point seven. And I have a link to that also in the Jupiter notebook. So it's not official official yet. It was just a discussion that happened, I guess, in the Python developer's email kind of threads. OK, so the first thing is to make sure that we're running the correct version of Python. So you should check that if you're not absolutely sure. So we're just going to import version info from this and then just look at what Vasilenko says. And in this case, you can see I'm running three point six point two. All right. So now let's see, what do I mean by the dictionary keys being having the order preserved? Let's say I have a dictionary and let's put two keys in that dictionary and I'm going to put them like B is one and A is two, OK? So now if I look at D dot items, let's say, or I can also look at D keys and dot values and then died of items if I type it correctly. So you'll notice that the key is on the order in which I specify them in the dictionary B and A, and the values are in the same order as well, and then the items are in the same order as well. Now let's say that I go ahead and add an item and I say, let's say X is three. Now I can look at these keys again, D values again and D items. And you can see that X shows up as the last key in the Keys. The three shows up as the last value and X comma three shows up as the last item. In other words, we're getting older resolved. Now what happens if I delete one? Let's say I'm going to delete a so I'm going to delete a OK and now I'm going to say delete A from D. OK, so now I'm just going to copy this actually I'm just going to look at items. That's good enough. Well if I can type it maybe I should pick something I can type. OK, so A is gone right. A was the second element is gone now what happens if I insulted back again. So I'm going to put back a as one and let's look at the items again and you can see A is at the end. So it does preserve that, you know that that order I removed a from the middle of the dictionary, I read it re added back in and added it to the end of the dictionary. So we have essentially inherent ordering in those dictionaries, hence why we don't need an order dict. Now I want to point out something that's kind of weird and it caught me by surprise. I wasn't aware of this because it doesn't do that in the Python interpreter, but Jupiter notebook plays around when you display your dictionary, so let's see what I mean. OK, so if we look at the items, you'll notice that we have B, X and A look. What happens if I just bring back D via Jupiter? I get A, then B, then X. So I think what it's doing, it's basically altering the keys, it's sorting the keys lexical graphically, alphabetically if you want. It's sorting those keys. In this case I've got string so lexicographical and then displaying it. So be careful. Do not use this as you do not use the output of just looking at the representation of D in Jupiter based on the output here that is now incorrect. Before it didn't really matter because we didn't have all the guaranteed, so we didn't really care how it was being displayed. And so it ordered the keys before displaying them, which was pretty nice. In fact, I think the P print function, the pretty print function does the same thing. The pre-print function will alter your keys before it pretty prints and. It was actually a fairly lengthy discussion about that and things got a little heated, but people were discussing whether pretty print should now honor the order of the dictionaries or whether it should, you know, sort the keys and then print out and it's pretty print. And so the consensus was finally that it will mean it will be backward compatible. It will sort the keys. So most likely Jupiter's doing something similar or maybe they're using pretty print. So we saw how we can add and remove items and it preserves the order. So let's look at our dictionary again. So again, I can do print D that's still fine print D will still keep and show the correct order. So you can see that B, X and A so as the last item. So if we say D dot pop item it pops the rightmost item. Right. It popped a which was that key here. Now if we prendes of course that value gone. OK, so this is kind of like a stack. Of course I wouldn't really recommend using a dictionary as a stack. You'll get probably a better performance using a list. So certainly not using a dictionary this. There's no real reason to do that. So now what I'm wondering, of course, is, well, how do I pick a random item from the dictionary? Because it's ordered and it's not like you could pick a random item from previous dictionaries because the keys were not truly randomly ordered. Right. There was hash values involved and it wasn't truly random. So I'll come back to that. Actually, I'll probably do a short video on how to pick a random item from a dictionary. Now, what about the update method? You know that you can update one dictionary using another dictionary. So what I mean by this. Well, let's go ahead and create this dictionary. A is one and B, let's say is two hundred. And let's call these two. Let's make that another dictionary. Let's say A is one hundred and D is three hundred and let's say C is four hundred. So I'm just keeping the DNC not ordered just so we can see if there's any kind of idiosyncracies with that. So here's D one and two. Now what you can do is you can say D one dot update D two and what it does essentially it merges these two into D one. So any existing key that will replace the value, any keys that are not in D one, but that R and D two will get merged into D one. So what I'm interested in knowing is does the order get preserved? And I'm guessing that when we merge these two into D one, it's going to replace A is going to stay in its first position and then D is going to get added in the third position and C is going to get added in the fourth position. In other words, the merge is going to respect the order of the keys in D two when it adds them to D one. So let's see. So we've done that. So this is an in-place mutation of D one. So now let's go ahead and print D one. Let's see what we get. And indeed we get that. So as you can see, the order was preserved. So that's great. All right, so now how about all the DECT, can we truly get rid of all addict? Well, old addict has a few methods that are not available in the regular DECT, for example, they move to and method and you specify the key that you want to move to the end and then you specify optionally what the end is. Is the end the beginning of the dictionary, or is the end the end of the dictionary? So is it the last item or not? So if you say last equals true, it's going to move it to the right side. The rightmost element of the dictionary in this ordered dict and if you say lastic false is going to move it to the beginning. And then similarly, there's also a pop item. So pop item also takes the last equals. True, and basically it's going to remove an item from the dictionary. Now we saw with the regular standard dictionary nine three point six, it just removes the last element. Well, that's what pop item does by default. But if you want to remove the first element, then you can say last equals false and then we'll remove the first element of the dictionary, not the last one. Again, if all you're looking for is some kind of list where you can add and remove items easily from both the front end and the end of the list, you should look at deck. That's in the collections module. OK, so those are two main items, it also supports revulsed, so you can actually pass reversed to reverse the iteration or you can call reversed on the order deck to reverse the iteration. So when you iterate in all the deck, normally you iterate from left to right. If you want to iterate from right to left, you just reversed instead. So let's take each one of those one by one and let's see if we can build an equivalent using just plain decks now in three point six. So the first one I want to look to is the move to end. So the move to end. Let's see if somehow we can do that, so let's start with a dictionary, so let's say A is one and B is two and C is three. So simple dictionary. Let me go ahead and print what our stout state is for the dictionary. Now, what do I want to do? I want to take an element. Let's say I want to take a and I want to remove it from the beginning and move it to the end. So essentially what I'm going to do is I'm going to pop it right. I'm going to pop the item with key. A pop is going to return the value. So now all I need to do is just set it to the key. And remember now, when I set the key because ordering is preserved, it should actually add it to the end of the dictionary. So in effect, what I've done here is moved a to the end of the dictionary. So that would be equivalent to move to end with. With just right that end with last equal to true, OK, that would be equivalent to that. So let's see now, let's go ahead and print moved a to end and let's just see what the state of the dictionary is. And yeah, that worked right away in the beginning was here and now a is at the end. So we can do at least the move to end lastic true of the old addict. Now how about move to front. OK, so by move to front what I mean is move to end last equals false. So let's see how we might do that. Now is not as easy because we want to put something in the beginning. We don't really have a way to do that with a standard deck. We can only basically add to the end, but we can't add to the beginning. So what we're going to have to do is we're going to let me start off with this dictionary and I'm just going to copy paste that from my notes. You can download the notebook. Let's start with this. And let's say that I want to move C to the beginning. So the first thing that I'm going to do, let me just show you kind of manually what I'm going to do. The first thing I'm going to do is I'm going to move C to the end, OK? So I'm going to take C and I'm going to move it to the end of the dictionary. That's going to be the first step. And then what am I going to do? I'm going to iterate in the dictionary. And now I know that I can iterate in an altered fashion. So I know I can start with the first key. I'm going to take that key. I'm going to pop it and move it to the end, and then I'm going to do the same thing again with B, I'm going to iterate again and pop it to the end and I'm going to do that again. I'm going to take that and move it to the end by popping and adding. And I'm going to do the same thing with y remove it and put it back at the end. And so now that I've done this, you'll notice that C is now in the beginning and then the remaining keys, A, B, X and Y have remained in their ordering. So that's the approach that I'm going to take here. So let's go ahead and print what the start is the start state of the dictionary D Now I'm going to pop and move basically so I'm going to C and then added back immediately to the dictionary. So that is now that has moved C to the end of the dictionary. OK, so let's go ahead and from that moved C to end. OK. So let's see if that works. Yeah. OK, so C is at the end, so we just use this technique here of move to end last equals true. Now, what we have to do is we have to iterate for I in range and we want to iterate from what? From the first element, this one up to but not including the last element. So we're going to go to the length minus one, essentially. So in terms of indexes, we're going to iterate from zero up to and including length minus two. So we're going to iterate up to the length of D minus one because we actually want to go to minus two inclusive and we're going to then take each one and move it to the end. So we'll do exactly what I showed you in the kind of manual process. So let's go ahead and do this. So I'm going to get an iterator on the keys. This is not a list. It's a view. So I need to get an iterator and I just want to get the first one right. That's all I'm interested in. I just want to get the first item in that all keys. So now that I have the key, I know what the key is. I need to move that item to the end of the dictionary so I can say Dovekie equals Deadshot pop key. OK, and let's go ahead and say print moved C to front. If this works correctly, we should have moved C to the front by essentially moving everything in front of C to the back. OK, and let's take a look. Yeah, we've got C in front and then you'll notice that everything else here, the order was preserved. So that's working great. All right. So the next one is pop. Last item. How are we going to do Pop? Last item? Well, that's really straightforward. Well, we have to do when we again copy this dictionary over here to pop the last item and we'll copy that as well to pop the last item. Will I have to say, because the item that's the default behavior. So top last item and you'll see that if I don't get a syntax error, you'll see that indeed we removed Y from the dictionary. OK, it's gone. OK, cool. Now how about Pop first item. And that one actually is quite easy as well. It's not going to be any more difficult almost than pop. Last item, we already know how to find the first key in the dictionary. So all we need to do I mean, copy and paste this again and all we need to do is just find the first key. Well, the key that we want to pop is going to be what while it's going to be, we first have to get our keys, OK? And there's different ways you can do it here. You could convert it to a list, OK, and then get the first item. You could do that. Or you could actually just use something like get the iterator on the keys and then just get the first one by calling next. So now we have the first key just to show you. That's where we get right. A is the first key. So we have that. And all we need to do, not just to say publicly, OK, and then we should have after the first item and as you can see, A is gone. So pretty neat, right? That's that's really neat. Um, so basically I'm not sure that we need an order dict anymore and it's probably going to remain in the standard library for backward compatibility, but most likely it's going to leverage. Now the dict is going to be a thin wrapper around a regular dict because most of the functionality is already there. And dict if you want to find out how the dictionary was actually implemented in this version of three six, then you can go to the website. It's really good. It's basically a python, a pure python, implementation of the dictionary of the new implementation of dictionary that Raymond Henningham did. And so he wrote this in Python to basically show as a proof of concept and to show how it works. Now, obviously, it's in the built in its in C Python is going to be written in C, but you can certainly see how it works and it's kind of standalone. And if you go to peepy and you try and take a look at it there, because that implementation is there almost that implementation is there, it's quite a bit more difficult to follow. This one is nice and self-contained, but I will do a video later on in the schools when I get into the section on dictionaries and we'll look at that in detail and see how it works. That will help us understand hash maps and things like that. All right, thanks for watching.